Voice application network platform

ABSTRACT

A distributed voice applications system includes a voice application rendering agent and at least one voice applications agent that is configured to provide voice applications to an individual user. A management system may control and direct the voice applications rendering agent to create voice applications that are personalized for individual users based on user characteristics, information about the environment in which the voice applications will be performed, prior user interactions and other information. The voice applications agent and components of customized voice applications may be resident on a local user device which includes a voice browser and speech recognition capabilities. The local device, voice applications rendering agent and management system may be interconnected via a communications network.

This application is a continuation of application Ser. No. 17/965,616,filed Oct. 13, 2022, which is a continuation of application Ser. No.17/410,683, filed Aug. 24, 2021, which is itself a continuation ofapplication Ser. No. 12/717,893, filed Mar. 4, 2010, which claimspriority to U.S. Provisional Application No. 61/157,337, filed Mar. 4,2009 and which also is a continuation-in-part of application Ser. No.11/514,116 filed Sep. 1, 2006, which itself claims the benefit of thefiling date of U.S. Provisional Patent Application No. 60/712,808, whichwas filed Sep. 1, 2005. The disclosure of all of the foregoingapplications is incorporated herein by reference.

BACKGROUND Field

The invention relates to systems and methods that utilize speechrecognition techniques to interact with a user to allow the user toobtain information and to perform various functions.

Background

There are various existing computer and telephony systems that providevoice services to users. These voice services can be speech recognitionand touchtone enabled. Examples of such services include voice mail,voice activated dialing, customer care services, and the provision ofaccess to Internet content via telephone.

One common example of a system that provides voice services is anInteractive Voice Response (IVR) system. In prior art systems, a userwould typically use a telephone to call in to a central computer systemwhich provides voice services via an IVR system. The IVR system deployedon the central computer system would then launch voice services, forinstance by playing an audio clip containing a menu of choices to theuser via the telephone line connection. The user could then make aselection by speaking a response. The spoken response would be receivedat the central computer system via the telephone line connection, andthe central computer system would interpret the spoken response usingspeech recognition techniques. Based on the user's response, the IVRsystem would then continue to perform application logic to take furtheraction. The further action could involve playing another menu of choicesto the user over the telephone line, obtaining and playing informationto the user, connecting the user to a third party or a live operator, orany of a wide range of other actions.

The ability to provide voice services has been quite limited by thenature of the systems that provide such services. In the known systemsthat provide voice services using relatively complex speech recognitionprocessing, the voice applications are performed on high end computingdevices located at a central location. Voice Application processingrequires a high end centralized computer system because these systemsare provisioned to support many simultaneous users. To get economies ofscale, it is imperative for these systems to share telecom and computingresources across users. Such high end computing systems share, acrossmultiple users, many phone lines, many IVR servers that connect to thephone lines, multiple speech recognition servers, one or moretext-to-speech servers, and a farm of application servers to processapplication logic during the course of a user interaction. Often, otherequipment like switches and media gateways are also present in thecentralized computer system. Management, integration and provisioning ofthese systems to support usage has been very complicated and expensive.Examples of such high end speech recognition systems are described inU.S. Pat. Nos. 6,229,880 and 6,741,677 to Reformato et al.; U.S. Pat.No. 6,891,932 and Patent Publication No. 2005/0053201 to Bhargava etal.; U.S. Pat. No. 6,477,240 to Lim et al.; and U.S. Patent PublicationNo. 2006/0015556 to Pounds et al., the respective disclosures of whichare all hereby incorporated by reference.

Because complex voice application processing must be provided using ahigh end computer system at a central location, and because users arealmost never co-located with the high end computer system, a user isalmost always connected to the central computer system via a telephonecall. The call could be made using a typical telephone or cell phoneover the PSTN, or the call might be placed via a VoIP-type (Skype, SIP)connection. Regardless, the user must establish a dedicated, persistentvoice connection to the central computer system to access the voiceservices.

FIG. 1 depicts a typical prior art architecture for a centralized voiceservices platform. In this type of system, the speech recognitionfunctions are performed at a central computer system. As shown in FIG. 1, a user telephone 1010 is used to place a telephone call to a centralvoice services platform 1060 via a telephone network 1040. The telephonenetwork 1040 could be a traditional PSTN, or a VoIP based system. Eitherway, the user would have to establish the telephone call to the centralvoice service platform 1060 via a telephone carrier.

As mentioned earlier, the central voice services platform must becapable of handling a large number of simultaneous telephone calls,especially during peak hours. Providing and maintaining the hardwarecapability to maintain multiple simultaneous separate voice telephonecalls is quite expensive. For instance, the average cost of providing asingle IVR telecom port presently ranges from $1,500 to $3,000 pertelephone line of service.

Merely paying for the connect time on a large number to telephone linescan be rather expensive. A public telephony based IVR system serviceprovider often must commit to a minimum volume of minutes with atelephony carrier vendor, leading to a fixed minimum telecom relatedexpense. This creates a situation where the service provider needs toquickly ramp up the volume of business in order to recover the telecomexpense per user, and thus increase the profit margin per user.

Also, as discussed, the central voice services platform is complicatedand expensive to begin with. These traditional IVR system deploymentsare also highly vulnerable to the failure of one or more components. Itrequires extensive redundant hardware and software systems in order toovercome this vulnerability in order to provide reliable service. Andbecause the hardware and software being used is expensive to begin with,providing redundant capabilities is very expensive.

Also, the prior art centralized voice services platforms, which dependon a telephony infrastructure for connection to users, are highlyinflexible from a deployment standpoint. The configurations of hardwareand software are all concentrated on a small number of high end servers.These configurations are technically complex and hard to monitor,manage, and change as business conditions dictate. Furthermore, thedeployment of existing IVR system architectures, and the subsequentprovisioning of users and voice applications to them, requires extensiveconfiguration management that is often performed manually. Also, changesin the configuration or deployment of IVR services within extant IVRarchitectures often require a full or partial suspension of serviceduring any reconfiguration or deployment effort.

The provisioning of a typical high end high end centralized computersystem has also been complicated by the type of voice services providedby such systems and the usage pattern associated with such voiceservices. For instance, a Voice Mail service system may have differentprovisioning requirements than an outbound notification system. In thisregard, the service provider using a high end high end centralizedcomputer system would have to manage a very high level of complexity ifit had to simultaneously provide contrasting voice services. The typesof voice services drive the traffic pattern of calls, driving the numberof phone lines needed, and the need for speech recognition servers andassociated application processing servers. These issues lead to manyspecialized voice services providers.

Further, cost structures and provisioning algorithms that provision thecapabilities of such a centralized voice services platform make itvirtually impossible to ensure that a caller can always access thesystem. If the system were configured with such a large number oftelephone line ports that all potential callers would always beconnected to access contrasting types of voice services, with differentand overlapping peak utilization hours, the cost of maintaining all thehardware and software elements would be prohibitive. Instead, suchcentralized voice services platforms are configured with a reasonablenumber of telephone ports that result in a cost-effective operatingstructure. The operator of the system must accept that callers maysometimes be refused access. Also, system users must accept that theywill not receive an “always on” service.

Prior art centralized voice services platforms also tend to be“operator-centric.” In other words, multiple different service providersprovide call-in voice services platforms, but each service providerusually maintains their own separate platform. Even when several serviceproviders are all using a common set of hardware and software, eachcompany usually maintains its own separate call in telephone number. Ifthe user has called in to a first company's voice services platform, hewould be unable to access the voice services of a second company'splatform. In order to access the second company's voice servicesplatform, the user must terminate his call to the first company, andthen place a new call to the second company's platform. Thus, obtainingaccess to multiple different IVR systems offered by different companiesis not convenient.

To address the problem of switching to a different voice servicesplatform, some IVR systems attempted to develop the ability to switch acaller off to a different voice services platform, or to a liveoperator, without forcing the user to hang up and place a new call.However, because a user is connected to the first voice servicesplatform via a dedicated telephone line connection, passing the calleroff to a live operator or to a third party's voice services platform canbe difficult and expensive. In some instances, it may be possible forthe central computer of the first voice services platform to communicatewith the PSTN to instruct the PSTN to re-connect the existing call to athird party number. But often the local PSTN carrying the call lacks theability to make such a switch. Even where it is possible, it isdifficult to develop communications switching code that will work withall PSTN equipment. More often, the central computer system is forced tomake a call to the live operator or third party voice services platformusing another dedicated phone line, and then bridge the original callerto the newly placed call to the operator/third party. The end result isthat the caller is now using two dedicated phone ports of the firstvoice services platform, and the user is no longer even making use ofthe first voice services platform. The operator of the first voiceservices platform must pay for the connect time on two dedicated lines,and the two dedicated lines cannot be used by the system to serviceother users.

In addition to the above-described drawbacks of the currentarchitecture, the shared nature of the servers in a centralized voiceservices platform limits the ability of the system to providepersonalized voice applications to individual users. Similarly, thearchitecture of prior art IVR systems limit personalization even forgroups of users. Because of these factors, the prior art systems havelimitations on their ability to dynamically account for individual userpreferences or dynamically personalize actual voice applications on thefly. This is so because it becomes very hard for a centralized system tocorrelate the user with their access devices and environment, to therebyoptimize a voice application that is tuned specifically for anindividual user. Further, most centralized systems simply lackuser-specific data.

The prior art systems, because they are so tied to the telephone networkto provide user access, have trouble rapidly deploying new applications.It becomes necessary to manage and re-route call traffic during anymaintenance activities. This can be particularly difficult with multiplecontrasting voice services being offered on the same system.

Some prior art voice services platforms were used to send audio messagesto users via their telephones. The central voice services platform wouldhave a pre-recorded audio message that needed to be played to multipleusers. The platform would call each of the users, and once connected toa user, would play the audio message. However, when it was necessary tocontact large numbers of users, it could take a considerable amount oftime to place all the calls. The number of simultaneous calls that canbe placed by the centralized voice services platform is obviouslylimited by the number of telephone ports it has. Further, in someinstances, the PSTN was incapable of simultaneously connecting calls onall the available line ports connected to the voice services platform.In other words, the operators found that when they were trying to make alarge number of outgoing calls on substantially all of their outgoinglines, the PSTN sometimes could not simultaneously connect all of thecalls to the called parties. Further, when a voice services platform isdelivering audio messages in this fashion, they tie up all the telephoneport capacity, which prevents users from calling in to use the service.

With the prior art voice services platforms, it was difficult to developefficient mechanisms for billing the users. Typically, the telephonecarrier employed by the user would bill the user for calls made to thevoice services platform. The amount of the charges could be determinedin many different ways. For instance, the telephone carrier could simplybill the user a flat rate for each call to the voice services platform.Alternatively, the telephone carrier could bill the user a per-minutecharge for being connected to the voice services platform. In stillother methods, the voice services platform could calculate user chargesand then inform the carrier about how much to bill the user. Regardlessof how the charges are calculated, it would still be necessary for thetelephony carrier to perform the billing, collect the money, and thenpay some amount to the voice service platform.

To begin with, these prior art billing mechanisms were cumbersome atbest. Both the telephony carrier and the voice services platform had tocreate relatively complex accounting systems to track the user'scharges, and to ensure that everybody received adequate payment for theservices delivered to the users.

Also, a voice services platform might offer a variety of differentservices, all of which are accessible once a caller has been connectedto the voice services platform. Some premium services might cost more todeliver to the user than simple standard services. Ideally, the usershould pay for the services that he uses. But in order to operate inthis fashion, it was necessary for the voice services platform to trackcharges on an individual, per-user basis, and to then inform the carrierof what to charge the user. This involves the cumbersome transfer ofbilling data, all of which had to be verified.

For all the above reasons, billing for services delivered to users ofcentral voice services platforms is cumbersome, expensive, and difficultto tailor to actual services usage.

Prior art voice services platforms also had security issues. In manyinstances, it was difficult to verify the identity of a caller. If thevoice services platform was configured to give the user confidentialinformation, or the ability to transfer or spend money, security becomesan important consideration.

Typically, when a call is received at the voice services platform, theonly information the voice services platform has about the call is acaller ID number. Unfortunately, the caller ID number can be falsified.Thus, even that small amount of information could not be used as areliable means of identifying the caller. For these reasons, callersattempting to access sensitive information or services were usuallyasked to provide identifying data that could be compared to a databaseof security information. While this helps, it still does not guaranteethat the caller is the intended user, since the identifying data couldbe provided by anybody.

The above references are incorporated by reference herein whereappropriate for appropriate teachings of additional or alternativedetails, features and/or technical background.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be described in detail with reference to thefollowing drawings, in which like reference numerals refer to likeelements, and wherein:

FIG. 1 is a diagram of a typical prior art system for providing speechenabled voice applications to a user;

FIG. 2A is a diagram of a system embodying the invention;

FIG. 2B is a diagram of another system embodying the invention;

FIG. 3 is a diagram illustrating elements of a Voice Applicationembodying the invention;

FIG. 4 is a diagram illustrating elements of a Voice ApplicationServices System embodying the invention;

FIG. 5 is a diagram illustrating a method embodying the invention forrendering voice applications;

FIG. 6 is a diagram that illustrates elements of a DVAES-enabled deviceembodying the invention;

FIG. 7 is a diagram of a method embodying the invention for providingcredentials to a DVAES-enabled device;

FIG. 8 is a diagram of a method embodying the invention for registeringa DVAES-enabled device;

FIG. 9 is a diagram illustrating elements of a Voice Application Agentembodying the invention;

FIG. 10 is a diagram illustrating a method embodying the invention forregistering a Voice Application Agent;

FIG. 11 is a diagram of a method embodying the invention for conductinga Voice Application with a user;

FIG. 12 is a diagram of elements of a Distributed Voice ApplicationManagement System embodying the invention;

FIG. 13 is a diagram of a method embodying the invention for registeringa new user;

FIG. 14 is a diagram of a method embodying the invention for associatinga user with a particular DVAES-enable device;

FIG. 15 is a diagram of a method embodying the invention for registeringa DVAES-enabled device, and providing the device with credentials;

FIG. 16 is a diagram illustrating a method embodying the invention forregistering a Voice Application Agent;

FIGS. 17A and 17B are diagrams used to illustrate how a VirtualDistributed Application Environment is defined;

FIG. 18 is a diagram illustrating a method embodying the invention forupdating information in a system embodying the invention; and

FIG. 19 is a diagram used to illustrate how voice browsers, voicesessions and audio and signaling sessions are interrelated.

DETAILED DESCRIPTION

Systems, devices and methods embodying the invention are intended toprovide users with speech- and touch tone enabled Voice Applications foraccessing various services. In this respect, the systems, devices andmethods embodying the invention serve some of the same functions as theprior art centralized voice services platforms.

However, unlike the prior art voice services platforms, systems andmethods embodying the invention utilize a highly distributed processingarchitecture to deliver the services. As will be explained below, theunderlying architecture and the distributed nature of systems andmethods embodying the invention allow the inventive systems to providethe same services as the prior art centralized voice services platforms,but with better performance, at a significantly reduced cost, and withfar fewer limitations. In addition, systems and methods embodying theinvention avoid or solve the drawbacks of the centralized voice servicesplatforms discussed above. Further, because of the way systems andmethods embodying the invention operate, they can provide new andadditional services that could never have been provided by the prior artcentralized voice services platforms. Systems and methods embodying theinvention also allow for much better personalization of deliveredservices, and they allow existing services to be upgraded, improved, orfurther personalized much more easily than was possible with the priorart centralized voice services platforms.

Systems and methods embodying the invention are intended to deliver orprovide Voice Applications (hereinafter, “VAs”) for a user. Beforebeginning a discussion of systems and methods that embody the invention,we should start by discussing what we mean by a VA, and what a VA can dofor a user. Unfortunately, this is somewhat difficult, because VAs cantake a wide variety of different forms, and can accomplish a widevariety of different tasks.

A VA provides a user with the ability to use their natural voice, touchtone sequences or other forms of user input, to access and/or control anapplication, to obtain information, to perform a certain function, or toaccomplish other tasks. Although the majority of the followingdescription assumes that a user will interact with a system embodyingthe invention, at least in part, via speech, other forms of userinteraction fall within the scope and spirit of the invention. Forinstance, developing technologies that allow a user to make selectionsfrom visual menus via hand or eye movements could also be the basis of auser interaction protocol. Likewise, developing technologies that areable to sense a user's brainwave patterns could form the basis of a userinteraction protocol. Thus, systems and methods embodying the inventionare not limited to speech-based user interfaces.

A VA could be specifically developed to utilize the benefits of speechrecognition-based input processing. For instance, a VA could bedeveloped to access, play and manipulate voice mail via speech commands.Alternatively, a VA could act as an extension or an enhancement oftraditional GUI-like applications to allow the traditional applicationsto be accessed and/or controlled by speech commands. For instance, a VAcould allow the user to call up specific e-mail messages on a displayvia spoken commands, and the user would then read the e-mail messages onthe display.

In some instances, a VA could act like one of the interactive voiceresponse systems that are accessible to users on prior art centralizedvoice services platforms. A VA could act in exactly the same way as aprior art IVR system to allow a user to obtain information or accomplishvarious functions using a speech enabled interface. However, because ofthe advantages of the new DVAES architecture, a system embodying theinvention can perform voice applications that would have been impossibleto perform on prior art centralized voice services platforms. Other VAscould perform a wide variety of other tasks. In most instances, the userwould be able to accomplish functions or obtain information by simplyspeaking voice commands.

With the above general description of a Voice Application (VA) asbackground, we will now provide an overview of systems and methodsembodying the invention. The following overview will make reference toFIG. 2A, which depicts a high-level diagram of how a system embodyingthe invention would be organized.

As shown in FIG. 2A, preferred embodiments of the invention would makeuse of an optional telephone network 230 and a data network 220. Thetelephone network 230 could be a traditional PSTN, a VoIP system, apeer-to-peer telephone network, a cellular telephone network, or anyother network that allows a user to place and receive telephone calls.The data network 220 could be the Internet, or possibly a private orinternal local area network or intranet.

In some instances, users would only be physically coupled to a datanetwork, such as the Internet. In this case, the user's on-siteequipment could enable them to place VoIP telephone calls via the datanetwork. Such VoIP telephone calls might make use of the PSTN, or theentire call might be handled over the data network. Regardless, inpreferred embodiments, the user would be capable of simultaneouslymaintaining a telephone connection and sending and receiving data.

Systems embodying the invention, as shown in FIGS. 2A and 2B, will bereferred to as having a Distributed Voice Application Execution SystemArchitecture (hereinafter, a “DVAESA”). Thus, the term DVAESA refers toa system and method of providing voice application services, in adistributed fashion, over a network, to a customer device. Such a systemis closely managed by a centralized system to, among other things,ensure optimum performance, availability and usability. In some of thedescriptions which follow, there are references to “DVAES-enabled”equipment or local devices/device. This means equipment and/or softwarewhich is configured to act as a component of a DVAESA embodying theinvention.

A user would utilize an audio interface device to access the DVAESA. Inthe embodiment shown in FIG. 2A, a first user's audio interface 200comprises a microphone and speaker. A second user audio interface 201comprises a telephone. The telephone 201 is also connected to the sameuser local device 210 as the first user audio interface. A third user'saudio interface 202 could also comprise a telephone. This telephone 202could be a regular wired telephone, a wireless telephone or even acellular telephone. The DVAES-enabled devices may support multiple audiointerface devices, and the multiple devices could all be of the sametype, or multiple different types of user audio interfaces could all beconnected to the same local device.

Each user would also make use of a local DVAES-enabled device that wouldact to deliver or provide VAs to the user through the user's audiointerface. The local DVAES-enabled devices would include a voice browsercapable of performing voice applications that have been distributed overthe network, some of which may have speech recognition functions. Suchvoice applications could be pre-delivered to the local DVAES-enableddevice, or the voice applications could be fetched in real time. Suchvoice applications are personalized to the user and optimized for thedevice. In the embodiment shown in FIG. 2A, each of the user localdevices 210, 212, 203 are coupled to the respective user audiointerfaces, and to the data network.

In some embodiments of the invention, a user audio device and aDVAES-enabled device could be integrated into a single electronicdevice. For instance, a PDA with cell phone capability could alsoincorporate all of the hardware and software elements necessary for thedevice to also act as the DVAES-enabled equipment. Thus, a single userdevice could function as both the DVAES-enabled equipment thatcommunicates with the network, and as the user audio interface. The userlocal device 203 shown in FIG. 2A is intended to illustrate this sort ofan embodiment.

Also, in FIG. 2A, various lines connect each of the individual elements.These lines are only intended to represent a functional connectionbetween the two devices. These lines could represent hard-wiredconnections, wireless connections, infrared communications, or via anyother communications medium that allows the devices to interact. Inaddition, in some instances the connections could be continuous, and inothers the connection could be intermittent. For instance, an audiointerface and a user local device could be located within a user'svehicle. In such a case, the local device within the vehicle might onlybe connected to the network through a cellular telephone network orthrough another type of wireless network when such connectivity isrequired to provide a user with services. In a similar embodiment, thelocal device in the user's vehicle might only link up to the networkwhen the vehicle is parked at the user's home, or some other location,where a wireless connection can be implemented.

Also, the user audio interface 202 shown in FIG. 2A could be a cellphone that is capable of interacting with the normal cellular telephonenetwork. However, the cellular telephone might also be capable ofinteracting with the user local device 212 via a wired or wirelessconnection. Further, the cellular telephone 202 might be configured suchthat it acts like a regular cellular telephone when the user is awayfrom home (and is not connected to the local device 212). But thecellular telephone might switch to a different operating mode when it isconnected to the local device 212 (when the user is at home), such thatall incoming calls to that cell phone are initially received andprocessed by the local device 212. The DVAESA also would include somenetwork-based elements. As shown in FIG. 2A, the network-based elementscould include a VA rendering agent 240, a network storage device 242 anda system manager 244. Each of these network-based elements would beconnected to the data network.

Also, although they would not technically be considered a part of theDVAESA, there might also be some third party service providers 250, 252which are also connected to the data network, and/or to the telephonenetwork. As explained below, the VAs may enable the users to interactwith such third party service providers via the data and telephonenetworks.

When a DVAESA as shown in FIG. 2A is configured, VAs would be “rendered”by the VA rendering agent 240, the output of the rendering process wouldbe rendered VAs. These rendered VAs may be stored on the Network StorageDevice 242, or be distributed or delivered to a DVAES-enabled Device.“Rendering” refers to a process in which a generic VA is personalizedfor a particular user and/or one or more particular DVAES-Devices togenerate Rendered VAs. The system manager 244 could instruct the VArendering agent 240 to render a VA for a particular user, or suchrendering request could originate from the DVAES-enabled Device. TheDVAESA network data storage element 242 could be used to store genericVAs, rendered VAs, or a wide variety of other data and resources (e.g.audio files, grammars etc).

As mentioned above, the VA rendering agent would personalize a genericVA during the rendering process. This could take into account personaltraits of the individual user, information about the configuration ofthe local device(s), or a wide variety of other things, as will beexplained in more detail below. The information used to personalize a VAduring the rendering process could be provided to the VA rendering agentat the time it is instructed to render the VA, or the VA rendering agentcould access the information from various data storage locationsavailable via the data network.

The user's local devices would typically be inexpensive computingdevices that are capable of running a voice browser and performingspeech recognition capable rendered VAs. In many instances, the localdevice would be physically present at the user's location, such as ahome or office. In other instances, however, the local device could be avirtual device that is capable of interacting with one or more useraudio interfaces. As mentioned above, the local devices may also storerendered VAs, and then act to perform the rendered VAs to the user'saudio interface. The user local device could be a customer premisedevice that is also used for some other function. For instance, thelocal device could be a cable modem or set-top box that is also used toconnect a television to a cable network, however, the device would alsobe configured to perform VAs for the user via the user's audiointerface.

In one simple embodiment of the invention, a local low-power device 212would be linked to a user's telephone 202. The local device 212 wouldalso be linked to the Internet 220 via a medium to high speedconnection, and possibly to the telephone network 230. The user couldspeak commands into the telephone 202, and those spoken commands wouldbe processed by the local device 212 to determine what the user isrequesting. In some instances, the local device 212 may be able tosatisfy the user's request. In other instances, the local device 212might need to request information from a VA Rendering Agent 240 tosatisfy the user's request. If that is the case, the local device 212would send a query over the data network 220 to the VA Rendering Agent240 for some type of content. The requested content would be returned tothe local device 212, and the local device 212 would then provide thecontent to the user via the user's telephone 202. In other instances,the local device may be able to query other network-connected elementswhich are not a part of the DVAES Architecture, and those other elementswould return the requested data to the local device so that the datacould be delivered to the user via the audio interface.

Depending on the VA being performed, the functions that are performed inresponse to a user request may not involve playing audio information tothe user via the user's audio interface. For instance, the local devicecould be performing a VA relating to accessing e-mail. In this instance,a user's spoken request could cause the local device to act in a mannerthat ultimately results in the user's e-mail messages being shown on adisplay screen. In this instance, although the user makes use of aspeech-based interface to obtain information and/or perform a certainfunction, the ultimate result is not the playback of audio, but ratherdisplay of an e-mail message.

The end result of a user request could take many other forms, such asthe local device causing a certain action to be taken. For instance, theuser might speak a request that causes the user's home air conditioningsystem, to be turned on. The list of possible actions that could beenabled by the local device is virtually endless. But the point is thatthe local device is able to provide a speech-enabled interface to theuser, via the audio interface, to allow the user to accomplish a task.

In another simple embodiment, the user might pick up his telephone 202and speak a request to be connected to another person's telephone. Thelocal device would interpret the user's spoken request, and then takesteps to place a telephone call to the person identified by the user.This might involve connecting the user via the telephone network 230, orconnecting the user to the requested party via a VoIP call placed overthe data network 220.

It is also worth noting that when a user is connected to the DVAESarchitecture, the VAs provided by the system can completely replace thedial tone that people have come to associate with their telephones. Themoment that a user picks up his telephone, he will be launched directlyinto a voice application that is provided by the system. In the past,this may have been technically possible, but it was always accomplishedby making use of the traditional phone system. For instance, one of theprior art centralized voice services platforms would have been capableof ensuring that the moment a user lifts his telephone, that user wasimmediately connected to a central voice services platform that wouldguide the remainder of the user's experience. But this was alwaysaccomplished by establishing an immediate voice channel between theuser's telephone and the central voice services platform. And toaccomplish that, it was necessary to involve the telephone carrier thatwould link the user's telephone to the voice services platform. Incontrast, with the DVAES architecture, one no longer needs to make anyuse of the telephone carriers to provide this sort of a service. And, asnoted above, the user can still be easily connected to the regulartelephone network if he needs to place a call.

In the same vein, in the past, whenever a user wanted to have a thirdparty service answer his telephone calls, as in traditional voice mailsystems, it was necessary to involve the carrier in routing such callsto a third party service. Now, when a call is made to the user'stelephone, the DVAES architecture makes it possible to answer the call,and take voice mail recordings, without any further involvement of thecarrier. Here again, the DVAES architecture makes it possible toeliminate the services of the telephone carrier.

In both the examples outlined above, the involvement of the carriernecessarily increased the cost of providing the voice services. Becausethe carrier can be eliminated, the same sorts of voice services can beprovided to a user for a significantly reduced price. And, as explainedbelow, the services can be delivered with greater performance and withnew and better features.

In a DVAESA embodying the invention, as outlined above, rendered VoiceApplication processing is performed on the local device and theassociated voice recognition functions in most cases may also beperformed on the local device. For this reason, there is no need toestablish a dedicated duplex audio link with a remote high end computer.Also, because the local low-power device is coupled to a data networksuch as the Internet, it can rapidly obtain Rendered Voice Applicationsand associated data from various remote sources in order to satisfy userrequests. For these reasons, the simple low-power local device allowsone to provide the user with speech recognition enabled VoiceApplications without the need to create and maintain a high end speechservice platform with multiple telephone line access equipment.

As noted above, the local device could also use the network to obtainaccess to various other physical elements to effect certain physicalactions, such as with the home air conditioner example given above. Inthis context, the other physical elements could be connected to thenetwork, or the local device could have a local connection to physicalelements that are also located on the user's premises. For instance, thelocal device could have a hard-wired or wireless connection to manydifferent elements in a user's home or office that allow the localdevice to control operations of the physical elements. In otherembodiments, the piece of physical equipment could act as the localdevice itself.

One obvious advantage of a DVAESA over prior art voice service platformsis that a DVAESA embodying the invention can provide VAs to userswithout any involvement of a PSTN, VoIP, Peer-Peer carrier. The instantthe user picks up his telephone handset, he will be interacting with theDVAESA, not the telephone system. A large number of VAs could beaccomplished without ever involving a telephone carrier as the VoiceApplication is delivered and provided on the local device. Because theuser can directly access the DVAESA without making a telephone call, theoperator of the DVAESA will not need to pay a telephone carrier in orderto provide the service to users.

As noted above, if the user wishes to place a telephone call, this canbe easily accomplished. But there is no need to use a telephone carrieras an intermediary between the user and the DVAESA. This has multiplepositive benefits.

Also, for a multitude of different reasons, a DVAESA will be lessexpensive to deploy and operate than the prior art central voiceservices platforms. To begin with, because the DVAESA can provideservices to users without a telephone link, the DVAESA operator nolonger need to purchase and maintain multiple telephone line ports intothe system.

Also, the types of equipment used by the DVAESA are inherently lessexpensive to deploy and manage than the equipment used in a centralvoice services platform. A DVAESA embodying the invention usesrelatively inexpensive network appliances that can be located anywhere,and that can be deliberately distributed over a wide area to enhancereliability of the system. In contrast, a central voice servicesplatform requires expensive and specialized telecom equipment liketelecom switches, VoIP Gateways, Media Gateways with DSP and TelephonyProcessing, IVR servers, and centralized ASR and TTS Engines. Thecentral voice services platforms also require more intensive managementand provisioning than a DVAESA, and this management must be provided byhighly skilled personnel as most of the equipment used is highlyproprietary in nature. In contrast, the DVAESA is largely managed by anautomated management system, as will be described in detail below. Aprior art central voice services platform is only able to simultaneouslyservice a limited number of users As noted above, in the prior artcentral voice services platforms, a dedicated voice link, via atelephone call, is maintained for each connected user. Once all linesare connected to users, no additional users are able to access thesystem. Hence the maximum number of simultaneous users that can besupported at any given time is equal to the lesser of the number ofaccess lines or the number of associated telephony/IVR ports an operatormaintains.

In contrast, a DVAESA embodying the invention has a very high limit onthe number of users that can be simultaneously serviced. In a DVAESAembodying the invention, the moment a customer picks up his telephone hewill be connected to the system. Thus, a DVAESA embodying the inventionis “always on.” Also, much of the interactions between the user and thesystem are handled directly by the local device on the customerpremises. If the local device cannot immediately service a user request,and additional information is needed, the local device may make ansynchronous or asynchronous request over the Internet. Typically, theinformation will be quite rapidly returned and played to the user. Thus,even if there is a small delay, the user is nevertheless still connectedto the voice services system.

With the DVAESA model, the same number of server assets can handle datarequests from a much larger number of users as compared to the prior artcentral voice services platform. This is also another reason why aDVAESA is less expensive to deploy and maintain than a prior art centralvoice services platform.

In addition to being easier and less expensive to deploy and maintain, aDVAESA embodying the invention can also scale up much more quickly andat a lower cost as new users are added to the system. To begin with,because the DVAESA does not require dedicated telephone lines tooperate, there is no cost associated with adding additional telephoneports to the system to accommodate additional users. Likewise, as newusers are added, there are no new additional telecommunications expensesfor more connect time or access. In addition, for the reasons notedabove, the equipment used by the system is far less expensive than theequipment used in a central voice services platform. Thus, adding anynew equipment and users is less expensive for a DVAESA. Moreover,because it requires less equipment to service the same number of usersin a DVAESA, there is much less equipment to purchase and maintain foreach additional 1000 users.

A DVAESA embodying the invention is inherently more reliable than aprior art central voice services platform. Because the assets of a priorart system are typically located in a few physical locations, poweroutages and other physical problems are more likely to prevent usersfrom being able to use the system. In contrast, a DVAESA can have itsequipment distributed over a much wider area to reduce these problems.The points of a failure of a DVAESA can be highly localized and it isvery cost effective to replicate DVAESA equipment.

Moreover, the underlying nature of the DVAESA makes it easy to connectmultiple redundant servers to the network, so that in the event one ormore assets fail, redundant assets can step in to take over thefunctions of the failed equipment. This was difficult to do in prior artcentral voice services platforms, and even when it was possible toprovide redundant capabilities, the cost of providing the redundantequipment was much higher than with a DVAESA.

In addition, a prior art central voice service platform needs atelephone carrier to provide access to the users. If the telephonecarrier has a service outage, the prior art system cannot function. Incontrast, a DVAESA does not have any reliance on a telephone carrier.

The only network required to provide the DVAESA is the data network likethe Internet. The user in most cases will not experience an interruptionto access to the voice services of a DVAESA, even if there is an outagethat disables the local device's access to the Internet. The localdevice could potentially perform some of the applications withoutconnecting to the network. This indicates that for some VoiceApplications in the DVAESA, it may be sufficient for the local device tohave intermittent access to the Internet.

The architecture of a DVAESA makes it inherently able to deliver certaintypes of VAs with vastly improved performance. To use one concreteexample, as noted above, when a central voice services application isattempting to deliver the same audio message to a large number of users,the central voice services application must place a telephone call toeach user, using a dedicated phone line, and deliver the message.Because the central voice services platform only has a limited number ofoutgoing lines, it can take a significant amount of time to place allthose calls.

In contrast, in a DVAESA embodying the invention, it is not necessary toplace any telephone calls to deliver the audio message to users.Instead, a server which is part of the system can push instructions toplay the audio message, and the message itself (the message could bestored in advance of when the event to deliver the message occurs), toeach of the local devices, and the local devices can then play themessages for each individual user. In variations on this theme, theserver might only send the instruction to play the message, along with areference to where a copy of the audio message is stored. Each localdevice could then download a copy of the message from the indicatedlocation and play it for the user. Regardless, it would be possible forthe DVAESA architecture to deliver the audio message to all the users ina small fraction of the time that it would take the prior art centralvoice services platform to accomplish the job.

Moreover, as also explained above, while the prior art central voiceservices platform is making calls to deliver audio messages to aplurality of users, it is tying up its phone lines, and thus itscapacity to allow users to call in for services. In contrast, when aDVAESA is delivering audio messages to a plurality of users, the usersare still able to access their voice services for other purposes.

As another example of how a DVAESA embodying the invention deliversbetter performance than prior art central voice services platforms,recall the earlier discussion about what happens when a user is loggedinto a first voice services platform, but wants his call to betransferred to a live operator, or a second provider's voice servicesplatform. It was costly to enact a transfer, and the transfer eitherrelied on features of the telephone carrier, or the first voice servicesplatform had to establish a second call, using a second dedicated line,and then bridge the two calls.

In contrast, if a user is interacting with a VA on a DVAESA embodyingthe invention then he is not yet connected to anything via the telephonenetwork. If the user wishes to be connected to a live operator, theDVAESA can simply setup an outgoing telephone call from the user'sphone. In fact, it might even be possible to connect the user to theoperator or third party voice service platform using the network. Thus,enacting this sort of transfer is far easier with a DVAESA as comparedto the prior art central voice services platform.

A DVAESA embodying the invention also makes it possible to deliver manynew voice applications and services that could never have been providedby the prior art central voice services platform. Several examples areprovided below. In most cases, it is the underlying differences in thearchitecture of a DVAESA embodying the invention, as compared to theprior art voice services platforms, which make these new servicespossible.

In the past, the user would either need a telecom provider or a voiceservices provider to intercept incoming calls and provide voice serviceslike an auto attendant, call screening, or voice messaging. Such callservices were possible, but were expensive (feature and toll charges)and complicated to operate as they needed the constant support of theservice provider (due to technology complexities). Such services alsousually required cumbersome touch tone sequences to enact and controlthe services. The DVAESA makes such services available to the user in avery cost effective manner, and the services can be easily controlledusing the local device powered by Voice Applications that aredistributed over the network.

In another example, the user could configure a voice application to runconstantly in the background, and then take a certain action upon theoccurrence of a specified event. So, for instance, the user could set upa voice application to break into an existing telephone conversation tonotify him if a particular stock's trading price crosses a threshold. Inthis scenario, the voice application would periodically check the stockprice. If the threshold is crossed, the voice application could causeany existing telephone call that the user is on to be temporarilysuspended, and the voice application would then play the notification.The voice application could then return the caller to his call. Thissort of a voice application would also be very complicated to provideunder the prior art central voice services platform.

The graceful integration of advertising messages is another example ofhow a DVAESA embodying the invention can provide services that wereimpossible to provide with prior art central voice service platforms. Asan example, if the user lifted the telephone and spoke a command thatasked for options about ordering a pizza, the system could respond witha prompt that said, “to be connected to Pizza Shop A, say one; to beconnected to Pizza Shop B, say two. By the way, Pizza Shop A is having atwo for one special today.” Thus, the advertising message could begracefully incorporated into the played response. Also, the advertisingmessage would be highly context relevant, which would make it moreinteresting to advertisers. Thus, advertising revenue could be collectedby the operator of the DVAESA system.

A DVAESA embodying the invention could also be used to rapidly collectdata from a very large number of users in ways that would have beenimpossible with prior art central voice services platforms. In thisexample, assume that a television program is currently airing, andduring the program, viewers are invited to vote on a particular issue.In prior art systems, the users would typically place a telephone callto a central voice services platform and make a voice vote. However, asnoted earlier, prior art voice services platforms are only able to talkto a limited number of callers at the same time because the callers mustbe connected by dedicated phone lines.

In a DVAESA embodying the invention, the user might be able to pick upthe phone and say, “I want to vote on issue X.” The system would alreadyknow that viewers of a television program had been invited to place avote, so the system could immediately take the user's voice vote. Thesystem could also tabulate the votes from all users making similar voicevotes, and then provide the voting results to the television showproducers in real time. Because so little actual information is beingexchanged, and the exchanges are made over the Internet, thousands, andperhaps even millions of votes could be received and tabulated in a veryshort period of time. This would have been impossible with prior artcentral voice services platforms. Furthermore, a DVAES can distribute afully featured voice application that not only plays the message, butfurther solicits feedback from the user, optionally tailors theinteraction with the user, and may record any user feedback orresponses. Furthermore, if the producers of the television show werewilling to pay a fee to the operator of the DVAESA, the system could beconfigured such that as soon as viewers are invited to cast a vote, andfor the duration of the voting period, anytime that a user of the DVAESApicks up his telephone to access the system, the system would firstrespond with the question, “would you like to vote on issue X?” Thiswould be yet another way to derive advertising or promotional revenuefrom the DVAESA.

There are countless other ways to exploit the architecture of a DVAESAembodying the invention to accomplish tasks and to perform VAs thatwould have been impossible using the prior art central voice servicesplatforms. The above examples are merely illustrative.

A DVAESA embodying the invention also allows for much greaterpersonalization of the voice applications themselves than was possiblewith prior art central voice services platforms. In addition, thearchitecture allows the users themselves to control many aspects of thispersonalization.

To begin with, as explained above, in a DVAESA a VA Rendering Agent isresponsible for customizing voice applications, and then delivering thecustomized voice applications to the local devices at the customersites. Thus, the basic architecture assumes that each user will receiveand run personalized versions of voice applications. This differencealone makes it much, much easier to provide users with personalizedvoice applications than prior art central voice services platforms.

The VA Rendering Agent could personalize a voice application to takeinto account many different things. For instance, the VA Rendering Agentcould access a database of user personal information to ensure that a VAtakes into account things like the user's name, his sex, age, home city,language and a variety of other personal information. The VA RenderingAgent could also access information about the capabilities of the localdevice at the customer's location that will be providing the VA, andpossibly also the type of audio interface that the user has connected tothe local device. The VA Rendering Agent could then ensure that thecustomized version of the VA that is provided to the user's local deviceis able to seamlessly and efficiently run on the local hardware andsoftware. The VA Rendering Agent could also take into account userpreferences that the user himself has specified. For instance, the VAcould be customized to play audio prompts with a certain type of voicespecified by the user.

Another important way that VAs could be personalized is by having theDVAESA track how the user is interacting with the system. For Example ifthe user has a certain type of accent or has a certain pattern of use orhas a certain type of background noise, the VA Rendering Agent couldtake these factors into account on an on going basis to ensure that thecustomized VAs that are sent to the user are tuned to the user. Thesystem might also note that whenever a three choice menu is played tothe user, the user always makes the third selection. In that case, theVA Rendering Agent might be directed to re-render the VA so that the VApresents the third option first, instead of last.

There are any number of other ways that VA's could be customized orpersonalized to take into account aspects of individual users. And thesecustomizations are easily and automatically accomplished by configuringthe VA Rendering Agents to automatically incorporate thesepersonalizations when delivering VAs for users. Because the DVAESA isconfigured so that each individual user may have his own versions ofVAs, preferably stored on his local devices cache, this personalizationis not difficult to accomplish. Such personalizations are complimentedby the continuous analytics process that is being performed on DVAESAdata. This data is collected during the on going functioning of thesystem and is provided by all DVAESA components. After collection, thedata is analyzed, and the results of the analysis are used tocontinuously tune and improve the functioning of the system on anindividual user-by-user basis.

A DVAESA also allows for better, more direct billing for delivery orusage of services. Because there is no telephone company acting as anintermediary, the operator of a DVAESA can directly bill users for useof the system. Also, the way the system is configured, the user canselect individual services, which are then provided to him by renderinga VA and loading it on the user's local equipment. Thus, the user cantailor his services to his liking, and the operator of the DVAESA has aneasy time tracking what services the user has. For all these reasons, itis much easier to bill the user for use of the services.

Another benefit that flows from the DVAESA model is the ability of auser to access services provided from two different DVAESA operators ona single piece of local equipment. As will be explained in more detailbelow, a first DVAESA operator could load a first set of VAs onto theuser's local equipment, and a second DVAESA operator could load a secondset of VAs onto the same piece of operator equipment. For instance, thefirst DVAESA operator could be one that provides the user with servicesrelated to his business, and the second DVAESA operator could be onethat provides the user with services relating to the user's personallife. There is no inherent conflict in both having two different sets ofVAs loaded onto the local device. And each DVAESA operator canthereafter maintain and update their respective VAs. Likewise, the usercan cause both sets of VAs to be loaded on a first device at his office,and a second device at his home. This allows the user to easily andimmediately access services from either operator, regardless of hispresent location. This sort of flexibility would also have beencompletely impossible in prior art central voice services platforms.

A DVAESA can also provide enhanced security measures compared to priorart central voice services platforms. For instance, because the DVAESAis interacting with the user via spoken commands, it would be possibleto verify the identity of a user via a voice print comparison.

In addition, and as will be explained in much greater detail below, theindividual local devices can be identified with unique ID numbers, andcredentials verifying the identity and permissions of users and devicescan all be created and stored in various locations on the system. Byusing these unique identification numbers and certification files, onecan ensure that only authorized users can access sensitive informationor perform sensitive functions.

The foregoing explanation has been focused on the differences between aDVAESA embodying the invention and a prior art central voice servicesplatform, and the inherent advantages that are offered by the DVAESA. Infact, there have been some prior attempts to provide speech recognitionenabled voice applications to a user via a device that is located in theuser's home or office. However, none of the prior art attempts evercontemplated the full distributed architecture of a DVAESA embodying theinvention, in which network assets cause personalized VAs to be renderedand delivered to local devices, and wherein an overall management agentcontrols the system assets.

In addition, most prior attempts focused on the use of speechrecognition software that was loaded onto a typical desktop PC or laptopcomputer. As will be explained immediately below, the use of PCs andlaptops has many inherent disadvantages which are overcome by a DVAESAembodying the invention.

To begin with, the cost of a PC or laptop computer is much greater thanthe projected cost of a local device embodying the invention. In manyembodiments of the invention, the local device will not include adisplay screen, a keyboard, speakers, or any of the other typicalperipheral devices associated with PCs. In addition, a local deviceembodying the invention need not be capable of performing any functionsother than the speech recognition enabled VAs. For all these reasons, alocal device embodying the invention can be produced and delivered to acustomer for a fraction of the cost of a PC or laptop computer, andassociated speech recognition software.

In addition, a simple local device embodying the invention is likely tobe far more reliable than a PC or laptop computer running specializedsoftware. As is obvious to those skilled in the art, a typical home oroffice PC is used for many different functions, is frequently used bymultiple different individuals, and it is exposed to all sorts ofpotential problems in the form of computer viruses and physical risks.As a result, the typical PC or laptop computer cannot provide the“always-on” type of reliability that we associate with telephones, cellphones and other simple computing devices. For these additional reasons,a simple local device embodying the invention will be much more reliablethan a PC or laptop running specialized software.

In addition, in the prior art systems, the user would typically need toload and configure the specialized speech recognition software onto hisPC before he could begin using speech enabled voice applications. Thisoften requires a relatively high degree of technical ability. Incontrast, and as will be explained below, a simple local deviceembodying the invention can be plugged into a telephone jack and anInternet connection, and associated systems supporting the local devicecan automatically handle all the required registration and configurationtasks. This sort of simple setup makes systems and methods embodying theinvention far easier to use than prior art systems that utilizespecialized software running on PCs or laptops.

Another place that speech enablement is found is in consumer electronicdevices. Examples could include cell phones that are capable ofhand-free dialing, or perhaps GPS devices that are used in vehicles andthat allow a user to control the GPS unit using voice commands. Hereagain, the DVAES Architecture embodying the invention also provides manybenefits and advantages compared to the speech recognition systems foundin these sorts of consumer electronic devices.

In consumer electronic devices, the speech recognition engines arenecessarily designed to be usable by a large number of different users.For this reason, they are designed to be usable by “the lowest commondenominator.” Basically, this means that the functionality must be veryeasy to understand, and that the voice prompts and voice inputs must bevery simple and predictable. In these devices, the user's responses aremerely compared to a simple list of acceptable responses. If there is nomatch, the device has no way of understanding the user's response or ofasking a different question to elicit more detail.

With the DVAES architecture, one can provide speech enabled consumerelectronic devices which have greatly expanded capabilities. To beginwith, as noted above, in a system embodying the invention, eachindividual user will be provided with a voice application that isspecifically tailored to their characteristics and needs. And no specialdevice training or complex setup procedures are required to provide thatcustomized voice application to each user. The VA Rendering Agentautomatically customizes the voice application when it is prepared andprovided to the user's local device. Also, even if the initial voiceapplication provided to the user is not efficient, as explained in moredetail below, a system embodying the invention has ways of reviewingusage history to automatically improve and replace the original voiceapplication with a new voice application that better serves the user'sneeds.

In addition, because the VAs are being rendered from a central renderingagent, the actual VAs can be more complex, and could be varied overtime. For instance, if the system notes that a first user only needs arelatively simple VA to interface with the device, the VA renderingagent could load a simple VA on the user's device. But if a second userneeds a more complex VA to effectively use the device, the system couldensure that a more complex VA with better and more detailed prompts isloaded onto the second user's device.

Furthermore, because of the network connection between the local deviceand the other elements of the system, such as the network data storageand other processing and data elements, the local device can provide amuch more complex and sophisticated voice recognition interface than ispossible with prior art consumer electronic devices. For instance, ifthe user's local device has difficulty interpreting a user response, thelocal device could enlist the aid of a more powerful speech recognitionengine on a network server to help interpret the response. Because ofthe greater sophistication that is possible with a system embodying theinvention, if the local device does not understand something, it canoften ask another question of the user to clarify the situation. Inaddition, the local device can offer a greatly expanded vocabulary andspeech processing by enlisting the assistance of network agents. For allthese reason, a consumer electronic device that is coupled into theDVAES architecture can provide a much more sophisticated voiceapplication than prior art devices which were not connected to anetwork.

In addition to the advantages described above, the inventors havedeveloped a comprehensive system which supports and in some measurecontrols the local device present in a user's home or office. Thecomprehensive system which stands behind the local device, and themultitude of advantages that it provides as compared to theabove-described prior art systems, will be fully described below.However, it is important to note that the Applicants are not aware ofany similar comprehensive system which is designed and configured tosupport a simple low-powered device that is located at a user's home oroffice, so that the low power device can interact with the user via acustomized speech enabled interface, and wherein that local low-powereddevice provides some or all of the actual speech recognitionfunctionality.

Having now provided a broad overview of the how a system embodying theinvention would operate, and the inherent advantages of a DVAESA systemas compared to prior art systems, we will now turn to a slightly morespecific description of the main elements of a DVAESA embodying theinvention, with reference to FIG. 2B. In doing so, we will introducesome new definitions and terminology which will be used throughout thefollowing detailed description. Once this more detailed overview iscomplete, we will then turn to a much more detailed description ofpreferred embodiments of each of the system elements, and descriptionsof preferred methods of accomplishing various functions.

A DVAESA would be configured to deploy and utilize one or more VoiceApplication Agents (hereinafter “VAAs”) which themselves enable thedelivery of a VA through a local device that would typically be locatedin a user's home or office. In some instances, a VAA may be whollyresident on a single local device. In other instances, the functions ofa VAA may be split between multiple portions of the overall system.Likewise, a single local device may only be hosting one VAA.Alternatively, a single local device may host multiple VAAs. Thesevariations, and the flexibility they provide, will be discussed in moredetail below. The important concept is that a VAA is the agent that isresponsible for delivering or performing a VA for the user.

The network 2130 shown in FIG. 2B could be the Internet. However, insome instances, the network 2130 could be a public or private localnetwork, a WAN, or a Local Area Network. In most instances, however, thenetwork 2130 will be the Internet. Also, the network 2130 could alsocomprise portions of the PSTN, existing cellular telephone networks,cable television networks, satellite networks, or any other system thatallows data to be communicated between connected assets.

The devices 2110 and 2120 appearing in FIG. 2B would be the local,relatively low-powered devices that are typically located at a user'shome or office. As shown in FIG. 2 , in some instances, a local device2110 could simply be connected to the user's existing telephone. Inother instances, the local device could be coupled to a speaker 2007 andmicrophone 2009 so that the local device can play audio to the user, andreceive spoken commands from the user. In still other embodiments, thelocal device may be a standalone telephone, or be included as part of acellular telephone, a computing device with wireless access, a PDA thatincorporates a cellular telephone, or some other type of mobile devicethat has access to a data network. Details about various local devicesand their capabilities will be provided below.

A system embodying the invention also includes components that delivervoice applications, data and other forms of content to the localdevices. These components could include one or more Voice ApplicationServices Systems (hereinafter VASSs). In the system depicted in FIG. 2B,there are two VASSs 2140 and 2150. A system embodying the inventioncould have only a single VASS, or could have multiple VASSs.

One of the primary functions of the VASSs is to render VAs and to thenprovide VA components to VAAs. In preferred embodiments, the VASS wouldprovide customized VAs components to VAAs, upon demand, so that the VAAscan perform the customized VAs components for the user. The VASSs couldpersonalize generic VAs based on known individual user characteristics,characteristics of the environment in which the VA components will beperformed, information about how a user has previously interacted withthe system, and a wide variety of factors. The VASS would thendistribute the personalized VAs components to the VAAs so that the VAAscan perform the VAs components for the users. The distribution of thepersonalized VA components to the VAAs could also be accomplished inmultiple different ways, as will be discussed below. A detaileddescription of the VASSs, their functions, and the multiple differentways that they can be configured into a system will be explained below.

A system embodying the invention may also include one or more ContentDistribution Services (hereinafter a “CDSs”). This is an optionalcomponent that basically serves as a storage and content distributionfacility. If a system embodying the invention includes one or more CDSs,the CDSs would typically provide network-based caching of content, suchas VA components, configurations, DVAESA components, and other shared orfrequently used content. The CDSs would be deployed throughout thenetwork to help reduce network traffic latency, which becomesparticularly noticeable in any speech interaction system.

The DVAESA components could broadly be identified as a Distributed VoiceApplication Execution System (hereinafter, a “DVAES”), and a DistributedVoice Application Management System (hereinafter, a “DVAMS”) A DVAEScomprises at least a VASS, one or more VAAs, and the underlying hardwareand software platforms.

The system shown in FIG. 2B includes a DVAMS. The DVAMS handles a widevariety of management functions which include registering users,specific items of hardware and other DVAES components, directing therendering, caching, distribution and updating of VAs components,organizing and optimizing the performance of system assets, and multipleother functions. The DVAMS may also include an interface that allows anindividual user to customize how the system will interact with him, andwhat products and services the user wishes to use. The DVAMS would alsoprovide an interface that allows system operators to manually controlvarious aspects of the system. Details above the DVAMS are providedbelow.

As mentioned above, individual VAs could be stored on a VASS. A VA isthen “rendered” by the VA generator of a VASS to produce personalizeddistributable VA components to one or more DVAES-enabled devices. Thisrendering process may make use of specific personal information relatingto an individual user. A database of user personal information could beaccessed by the VA or the VASS during the rendering process to allow therendered VA components to be highly personalized to the user. Thepersonalized VA components would then be distributed to a VAA located onone or more pieces of DVAES-enabled equipment which will be accessed bythe user.

FIG. 3 shows an illustration of an exemplary VA. The VA includes a VASSinterface 18110, a dialog engine 18120, a dialog controller 18130, VAconfiguration materials 18140, an optional template 18150, a set ofexecution requirements 18160 and an optional manifest 18170.

The VA configuration materials comprise configuration data createdduring the deployment process. The VA configuration materials couldinclude a deployment specification, which may include life cycleinformation for the VA, rendering constraints, rules, distributable VAcomponents, and possibly a specification of the VASS storage location.The VA configuration materials could also include a dialog list, adialog output specification (e.g., SALT, VoiceXML, or others), andoptional dialog resources such as audio text, grammars and templates.The configuration materials might also include instructions specifyingthe flow between dialogs.

The execution requirements provide a description of requirements forexecution of a VA during rendering and performance. For example, theexecution requirements may name one or more data services that must beavailable during rendering, or may indicate whether the rendered VAcomponents require access to a high-speed data connection duringperformance. The VASS Interface 18110 provides access to a VA's internalmethods for rendering parts of specific VA components to the VASS.Examples of methods of the VASS interface that are provided to the VASSinclude generating full or partial VA components, pause generation,abort generation, and clean up generation. The VASS interface could alsoprovide data access services to the VA. This could include data accessto user profile data, DVAES performance data, application data andexternal data.

The VASS interface also provides the VA the capability to log detailedVA-specific events to the VASS OA&M Service. The VA Log items couldinclude details of the VA rendering process, such as start time, endtime, VA ID, and a Data Access Attributes component list. Additionally,the VA logs may include detailed error messages. The logging levelprovided by a VA may be configurable and may provide varying levels oflogging in response to a configuration setting. A VA may also performerror notifications to the VASS OA&M Service. The error notificationsmay be configurable and may provide varying levels of error handling inresponse to a configuration setting.

The VA dialog controller 18130 is the component that receives theinstruction to render the VA from the VASS interface. A dialog is a unitof interaction with the user. In its simplest form, a dialog could bethe ability to play an audio file such as a greeting prompt. In morecomplex forms, a dialog could be the sequence of instructions, alongwith a speech recognition context specification (such as a. grammar oran n-best List). A dialog in the context of the VA is a dialogspecification that defines what the dialog intends to perform, with nospecific instructions about how the dialog will eventually be performedby a VAA. The dialog specification is Voice Browser or VAA agnostic.

The dialog controller is aware of all the dialogs, the flow, and theassociated logic between dialogs in the VA based upon the VAconfiguration materials 18140. The dialog controller receives from theVASS partial or complete VA dialog rendering instructions. Based on suchinstructions, the dialog controller generates the specified dialogs withthe help of the dialog engine 18120. The dialog controller also receivesall the data from the VASS required by the dialog engine to produce apersonalized distributable VA component. The process of data collectionis performed by the VA generator.

The Dialog Engine 18120 of the VA is responsible for the creation ofpersonalized and distributable VA components. The dialog engine receivesan instruction from the dialog controller to render a specific dialog.Based on such an instruction, the dialog engine loads the dialogspecification and begins to render the dialog. The rendering processbinds a resource with a dialog. For instance, this could meanassociating a prompt file with a dialog like a “welcome greeting.” Thiscould also be associating a grammar with the dialog, such as a list ofnames for a voice dialing VA. The association of the resource is done bythe dialog controller based on rules and by accessing the VASS Dataaccess interface.

In some cases, the dialog engine may create the resource. Some of theresources that could be created by the dialog engine include synthesizedspeech with the support of a TTS engine, compiled grammars based on thesupport of an ASR engine, and concatenation of audio files.

The dialog rendering process also includes transforming the dialog tothe specified output format. As an example, the dialog engine, based onVA configuration materials, may render a distributable VA component in aspecified format. In one embodiment, this distributable VA componentcould be rendered in accordance with the Voice XML specification.Alternatively, the VA component may be rendered in accordance with theSALT specification.

The dialog engine may use a template specification in the targetstandard as specified by the VA configuration materials, and completethe rendering once appropriate resources are generated and associatedwith the template. Upon completion of the rendering, the dialog enginegenerates an output that is in the form of a personalized distributableVA component. Such output is stored in the VASS storage, as instructedby the dialog controller. Once the dialog is “rendered,” the dialogengine informs the dialog controller and waits for an instruction toproduce the next dialog.

As mentioned, VA components are generated as an output of the VASSrendering process. VA components could be one of multiple differenttypes. For instance, a VA component could be a “resource component” oran “interaction component.”

Resource components are typically the most atomic form of VA components.Examples of resource components include audio files, and speechrecognition grammars, and ECMA script segments/files. Resourcecomponents may be generated by the VASS. For instance, the TTS enginecould generate an audio file, or a compiled grammar could be generatedby an ASR Service. Alternatively, resource components may be provided bythe operator. For instance, the operator could provide audio files.

Interaction components are logical constructs of instructions for a VAAto enable performance. Interaction components enable interactions withthe Users. A simple interaction component could instruct the VAA to“Play an audio file.” A complex interaction component could instruct theVAA to capture a User personal identification number, and then validatethe spoken input. Interaction components reference resource componentsto successfully perform a VA. A VA interaction component instructionconstruct may conform to a standard specification, for instance VoiceXMLor SALT. An example of such a construct is the <Prompt=foo.wav> Tag inVoiceXML that would instruct the VAA to play the Foo.Wav audio file as aprompt.

The optional manifest provides a description of the VA and itsconstituent pieces, and may include descriptions and cryptographichashes for rendered VA components for use by cryptographic integritychecking algorithms.

With the foregoing overview, the discussion will now turn to detailsabout many of the individual components of the system. Once theindividual components have been explained, the discussion will return toa more global perspective, and the advantages of the DVAES architecturewill be reviewed.

The Voice Application Services System

A Voice Application Services System (VASS) provides the combinations ofsoftware components that collaboratively enable the rendering of VoiceApplications. The VASS Hardware could include proxy servers, webservers, application servers, load balancers, and other commerciallyavailable systems that are integrated to enable the rendering of VoiceApplications. Foundation software for a VASS includes standard serverbased computing technologies such as Dot-Net, PHP, and Java.

As explained above, VAs are deployed on a VASS. The VASS uses thedeployed VAs to render one or more VA Components. These VA Componentsare personalized and are distributable to one or more DVAES-enableddevices to be performed by a VAA.

FIG. 4 illustrates one embodiment of a VASS 9000. The VASS comprises aVoice Application Generator 9210, a Data Access Service 9220, a ContentDistribution Manager 9310, a Voice Application Manager 9420, a webservice component 9410, an optional cache subsystem 9110, a VoiceApplication Storage 9320, and an OA&M Service 9500.

The VA generator 9210 implements the “rendering” process by whichdistributable VA components are created. It takes instructions andconfiguration parameters from one or more DVAMSs, combines them withthird party information and user-specific personalization instructions,and produces a set of distributable VA components for use by a VAAloaded on one or more DVAES-enabled devices. The Data Access Service9220 may provide an interface between the Voice Application Generatorand a plurality of data sources, including DVAES-specific sources forpersonalization information, and public data sources.

The VASS could maintain and manage access to several data sources toenable aspects of the DVAES. The rendering process may require thatsystems external to the VASS be referenced and materials collected fromone or more data sources for use in the rendering process. The DataAccess Service provides the data collection and interface services thatcollect information from one or more data sources, reformats, ifnecessary, to a form usable by the VA generator, and then provides thosematerials to the VA generator. Information collected by the Data AccessService could fall into multiple different categories.

The information collected by the Data Access Service could include userprofile data, user configuration information, and user-definedapplication specific preferences. User configuration informationincludes such data as account number, zip code, VDAE association(s), andallocated VAs. Some examples of user-defined application preferencesinclude attributes of an address book for voice application dialing, andhandling characteristics for re-direction to voice mail.

The information collected by the Data Access Service would also includeperformance data that is application-specific, VAA specific, environmentspecific, and configuration specific. The performance data could alsoinclude monitoring results and logging results. Application data isinformation collected or analysis results associated with the use of aVA in order to self-configure and operate effectively. The applicationdata could also comprises volatile application specific information,such as the current number of unread emails, the number of new voicemail messages, a reminder in a calendar application, or informationabout specific reservations (for a reservation management VA).Application data may be obtained from one or more DVAMSs, VAAs, or othersystems. Furthermore, the types of information in the DVAMS and in theDVAES-enabled device may be different, in that the DVAMS information mayhave had the above referenced analysis step performed on it, and theDVAES-enabled device data may comprise raw VA logs from a voice browser.

The Data Access Service also provides an interface to external dataservices to obtain external information. These services may includestock quotes, external email systems, information clipping services, andthe like. In other exemplary embodiments, the Data Access Service mayprovide information from common information sources, such as news wiresand RSS feeds.

For example, if a VA that is to be rendered is to provide a user anear-real time stock quote whenever they access the system, the VAgenerator recognizes the request for a stock quote while processing theVA, and personalizes the request using the user's specified stock symbolspecified in the user preferences. The VA generator then requests apiece of content from the Data Access Service that represents thecurrent stock price of the stock represented by the user's symbol. TheData Access Service obtains that information and provides a VA componentto the VA generator that comprises the current stock price. The VAgenerator adds the reference to the Data Access Service component to thepersonalized VA, and causes the VA to be published. The Data AccessService, meanwhile, publishes the VA component to VA storage and startsa process in which the Data Access Service regularly updates the VAcomponent with the current stock price and republishes the component foruse by the DVAES.

The Data Access Service is especially useful in that some of therequired content is stored in differing locations and in differentformats. For example, user preference materials may be stored in a localdatabase, cache, or directory of the registry component of a DVAMS,while dialog personalization materials may be scattered between aplurality of DVAMS and DVAES-enabled devices. The Data Access Servicehas a data transform capability in which it can transform the format ofinformation from one format to another, and may optionally call externalservices to assist with the transforming. In the above example, the DataAccess Service might call an external service provided by a DVAMS toanalyze a user's raw logs obtained from a DVAES-enabled device in orderto determine specific types of personalization required and to producethe necessary personalization information to enable the VA generator topersonalize an aspect of a VA.

The VA Storage 9320 is local or network storage for rendered VAcomponents. The VA storage is accessed by the VA generator and by theDVAMS. The VA storage in combination with the content distributionmanager 9310 and/or the DVAMS are responsible for pushing rendered VAcomponents to a content distribution service (CDS) or to individualDVAES-enabled devices. The VA storage is also accessed by the CDS orindividual devices that may request content from the VA storage.

The Voice Application Manager 9420 and its associated web services 9410provide a service by which un-rendered VAs may be provided to a VASS.These components are discussed in more detail below. The VoiceApplication Manager and its associated web portal interface provides forthe collection and management of voice application configurationinformation. The VAM may include a web portal interface, a database, adirectory or registry in which to store the application configurationinformation, and application logic to manage access to voice applicationconfiguration information. These types of data storage mechanism arewell understood to those skilled in the art.

In at least one embodiment, the Data Access Service may make requests ofthe VAM for user preference and application configuration information insupport of the VA generator. Additionally, the VA generator may alsomake requests of the VAM for needed personalization materials. Theserequests may be made directly to the VAM, or may be made using the webservices portal provided by the VAM.

The optional cache subsystem 9110 provides a cache for obtaininginformation published by other VASSs, DVAMSs, or DVAES-enabled devices.

The Voice Application Storage 9320 is the primary storage for renderedvoice application components prior to and post distribution. Voiceapplication storage may be a disk store, a directory service, a contentmanager system, or other mechanism, as are well known in the art forstoring voice application components in a manner in which they may bestored and retrieved by other system components.

Voice Application Storage optionally exposes an external interface so itmay be accessed from other systems. The external interface may be aservice (such a WEBDAV), a directory (such as LDAP), a database, orother method well understood to those skilled in the art.

A VASS may “render” a VA by producing a version of VA components thatare customized to operate within the constraints of each DVAES-enableddevice that the VA is allocated to. For example, rendering may generateVA components that take into account DVAES-enabled device limitations,such as system memory, the version of a VAA available on the device,historical or projected network performance, or other DVAES-enableddevice factors.

The VASS may further customize each instance of the VA components withpersonalization information associated with a specific user or group ofusers. The personalization of VA components may also take into account.For example, if a specific voice/user interface paradigm (for instanceuser skill level=assisted, or user skill level=advances), or otherpersonalization preference is specified, these changes are applied tothe specified instance(s) of the rendered VA components. Also, userusage and behavior trends, and speech recognition grammar tuning resultsmay also be taken into account during the rendering process topersonalize the VA components.

Finally, the VASS makes the rendered VA components available to one ormore DVAES-enabled devices, and facilitates the distribution of VAcomponents through the CDS and caching components of the VASS andDVAES-enabled device to maintain the currency of the VASS components ateach DVAES-enabled devices.

The VA generator could be activated by a VASS event listener, by theDVAMS or by a VAA request. The VASS event listener is a service thatactively listens or monitors for events in DVAES and the DVAMS. The VASSevent listener detects the occurrence of an event and initiates therendering process. Alternatively, the DVAMS initiates the renderingprocess upon the occurrence of an event in a DVAES and/or the DVAMS. Insome circumstances, an external service such as a VAA, could initiatethe rendering process directly by issuing a request over the network.

The rendering process could be partial or complete. During partialrendering, only select voice application components could be rendered.The specific voice application components that are rendered are afunction of the impact area based on a DVAES or a DVAMS event. Completerendering is a process by which all components of a voice applicationare rendered. The motivation for rendering could be the creation of anew user or a configuration change in a device.

When a VA generator is notified of a change in the DVAES and/or theDVAMS, the generator loads the VA from the VA storage and executes thegeneration process for the VA to generate personalized instances of theVA components for distribution. If disparate devices require differentrenderings, the VA generator produces as many rendered instances of theVA components as required.

An example rendering process is shown in FIG. 5 . In this process, aVASS is notified that something has changed, and that it is necessary torender a VA.

As shown therein, in step 10110, the VA generator receives anotification indicating that an aspect of data or content has changed.In step 10120, the VA generator collects relevant DVAMS, DVAES and VAconfiguration materials, and user specific personalization materialsfrom various sources by calling the data access service of the VASS toobtain the materials. In some cases, the materials are already availableon the VASS (e.g. in a local database, or in a cache). In other cases,the data access service must contact the appropriate service, database,or registry to obtain them.

Once the materials are collected, in step 10130 the VA generatordetermines the changed materials, and by inspecting the changedmaterials, determines the scope of the changes. After determining thescope of the changes, the VA generator determines the VAs that must berendered in order to provide rendered VA components to the DVAES inaccordance with the configuration and personalization materials.

Next, in step 10140 the VA generator loads the un-rendered VA from VAstorage that is relevant to the notification. For each rendering of a VAthat is required based upon the above change determination, the VAgenerator may take into account user preference materials, historicalrecognition performance, previous dialog interactions, and variousdevice and environmental factors (e.g., network quality, backgroundnoise) associated with the environment(s) in which the VA is expected tobe performed.

Then, in step 10150 the VA generator tailors a VA by associating theuser's preference materials with a VA. The user preference materials maybe used to adjust aspects of specific VA components, such as dialogs,language, audio, synthesized speech, and a variety of other items. Inthis way, a user may select the “sound and feel” of the voiceapplications that they are presented with. The user preference materialsare obtained by the VA generator using the VASS's data access service,which in turn obtains the materials from the DVAMS. Alternatively, thepersonalization materials may be stored in one or more devices, in theCDS, or in an Internet application. The data access service locatesthese materials and makes them available to the VA generator.

The VA generator may then initiate the rendering of personalized VAcomponents in one or more of three optional steps. The VA generatorinteracts with the dialog controller of the VA to perform each of thepersonalizations and to produce VA components. Personalization of VAcomponents using these materials may be a multi-part process. In someembodiments, the VA generator performs both parts of the process as oneoperation. In other embodiments; the steps are performed in two distinctoperations. Generally, the operations are one of collecting useractivity information that relates to a user's prior interactions with aDVAES, and performing a first operational analysis upon this informationto determine patterns of use, patterns of failure, and/or patterns ofinteraction. Then, a second operation is performed in which the VA ispersonalized based upon the results of the analysis. In this exampleembodiment, the process assumes that the analysis operation is alreadyperformed and the resulting personalization materials are made availableto the VA generator.

For instance, the VA generator may create personalized VA componentsusing recognition materials in step 10160. This results in a VA that hasbeen personalized to account for prior experiences of a user with theASR services available upon a user's DVAES-enabled device. Recognitionpersonalization uses information collected from previous interactionswith a user that identifies how a user has interacted with the ASRcomponents in the past, and may include the specification of alternateASR services, different grammars, or different recognition techniques.

In some embodiments of the invention, the VA generator requests therecognition personalization materials from a VASS's data access service,which in turn obtains the materials from information published by aDVAMS to a database, directory, or registry. In other embodiments, therecognition personalization materials are published from a DVAMS and arestored in a CDS, and may be obtained directly from a cache. In stillother embodiments, the recognition materials are stored in aDVAES-enabled device and are obtained directly from the device by theVASS's data access service.

The VA generator may also optionally generate personalized VA componentsusing dialog personalization materials in step 10170. This results in VAcomponents that have been personalized to account for prior interactionsbetween the user and the VA. Dialog Personalization can result inchanges in the Voice User interface and the dialog flow of the VA.

Dialog personalization uses information collected from previousinteractions with a user that identifies how a user has interacted withdialogs contained in VA components in the past. For example, the dialogpersonalization information may take into account the fact that in thepast, a particular user always selects the third option in a list ofinformation. Based on this fact, the order in which the options orinformation are presented can be changed. For example, if the user ispresented with “you have 5 appointments, 2 emails, and 2 voice mails”,and the user regularly selects the voice mail option, the dialog may beregenerated to present “you have 2 voice mails, 2 emails, and 5appointments,” or even “you have 2 voice mails and some other items,voice mail 1 is . . . ”.

In some embodiments, the VA generator may request the dialogpersonalization materials from a VASS's data access service, which inturn obtains the materials from information published by a DVAMS to adatabase, directory, or registry. In other embodiments, the dialogpersonalization materials are published from a DVAMS to other DVAESsystems and may be stored in a CDS, may be obtained directly from acache, or may be obtained from one or more DVAES systems. In otherembodiments, the dialog personalization materials are stored in aDVAES-enabled device and are obtained directly from the device by theVASS's data access service.

Further, the VA generator may also optionally generate personalized VAcomponents with environmental personalization materials in step 10180.This results in VA components that have been additionally personalizedto account for factors related to the anticipated performanceenvironment. Aspects of the DVAES-enabled device(s) and VAA(s) to whichthe personalized VA components will be distributed are considered inthis step.

Environment personalization materials use information collected aboutthe environment that the personalized VA components will likely beperformed in. These materials include VAA and DVAES-enabled deviceconfiguration information and other materials regarding network latency.These materials are used to adjust requirements for services that maynot be available in specific devices and to determine which portions ofa VA's components are already available without additional publishing ofthe components.

In step 10190, the VA generator then publishes the personalized VAcomponents to VA storage or forwards it to the VAA. The completelypersonalized VA components are now considered “rendered” and they arepublished to VA storage or delivered directly to the VAA. Publishing toVA storage involves making a local copy of the rendered VA components inVA storage and informing the DVAMS and the Content Distribution Managerthat an updated VA component is now available.

In some instances, it may be necessary to send rendered VA components tomultiple different VAAs. This could occur when a user has multipledifferent local devices, and the user wants to be able to access his VAsfrom each of the multiple local devices. In this situation, the DVAMSwould ensure that the rendered VA components are sent to all the user'sVAAs on all the user's local devices.

In step 10200, the VA Generator determines if there are more VAsrequiring rendering. If so, the process returns to step 10140, in whichthe next set of VA and personalization materials are selected forrendering. If not, the process terminates.

The Operations, Administration, and Monitoring service 9500 of the VASSis responsible for ensuring that the VASS components are workingefficiently. The OA&M Service is also the DVAES component thatinterfaces with the DVAMS. The VASS's OA&M service provides servicessimilar to the OA&M service of a VAA.

Upon start up, the OA&M Service loads configuration materials andestablishes a connection with the DVAMS. The OA&M service could operatein an active mode and/or in a passive mode. In active mode, the OA&Mservice starts all the other services in the VASS based on the orderspecified in the configuration data. In passive mode, all the VASSServices self-start based on a startup routine in the OS. Once theservices have started, they register with the OA&M.

The interface between the OA&M Service and the VASS services may bebased on an API or a messaging protocol. Examples of messaging protocolsthat may be used include SNMP, RPC, SOAP, and TCP/IP Messaging. Theconnection between the OA&M service and the DVAMS may also be based on anetwork provisioning, communication, and monitoring protocols like SNMPor tr-069.

The OA& M Service, based on an external instruction, such as one from aDVAMS, a DVAES-enabled device, or service condition, may shutdown andrestart the VASS components and services. Device and service conditionsinclude such items as CPU load, available memory, and changes inconfiguration. The OA&M service may notify services to reload changedconfigurations as an alternative to a service shutdown and restart.

The OA&M Service may provide a heartbeat service in deployments thatrequire one.

The OA& M Service may receive and store log and error events receivedfrom the VASS components and services. The OA& M service may propagatesuch log and error information to the DVAMS and optionally to anadditional network management system. Additionally the OA&M service maysend a health heartbeat signal to the DVAMS.

The VASS may be implemented as a single system, or the components of aVASS may be distributed across a plurality of systems to provideredundancy and the necessary performance. Accordingly, each component ofthe VASS must register with a DVAMS before it may be used within aDVAES, and each VASS component reports its current operating status to aDVAMS on a periodic basis.

Each VASS component is registered with at least one DVAMS when it isfirst integrated as part of a DVAES. Registration is preferablyautomatic and performed at the component level when the VASS componentis first instantiated. Once the address of a DVAMS is established by aVASS component, the VASS component connects to the registration servicepublished by the DVAMS and registers itself. Preferably, the VASScomponent provides a unique machine ID along with IP address and portinformation as part of the registration process. Authenticationoptionally may be required in order to permit registration.Authentication techniques are well understood to those skilled in theart and may include hardware certificates, account/passwords, or othermechanisms. Once satisfied that the request is both valid and should becompleted, the DVAMS updates its registration directory with theinformation provided by the VASS component, and enables the VASScomponent for use.

Each VASS component may be registered with multiple DVAMS systems. Insome embodiments, each VASS component may be registered with a DVAMSassociated with a specific DVAES. In other embodiments, where there is aplurality of DVAES implementations (for example, where two different,competing vendors have deployed DVAES architectures), a VASS componentmay register with separate DVAMSs associated with each of the DVAESdeployments.

DVAES-Enabled Devices

The local devices that will typically be used at a user's home or officewould be configured to host VAAs which perform VA components. In thisrespect, a DVAES-enabled device would typically be considered CustomerPremise Equipment. Saying that a VAA deployed on the DVAES-enabledequipment performs VA components means the ordered processing of VAcomponents, in which audio input is received and processed, and/or audiooutput is produced and/or actions are taken in accordance with theperformance of voice application components.

One or more VAAs could be deployed on a single DVAES-enabled device.Also, in some instances, the functions of a single VAA might be sharedbetween two or more such devices. The establishment and configuration ofthe VAAs on the DVAES-enabled equipment would be controlled primarily bythe DVAMS, as explained in detail below.

In some embodiments of the invention, the DVAES-enabled device could bea dedicated network appliance, similar to a cable set-top box or a cablemodem. In other instances, the DVAES-enabled device could act as boththe host for one or more VAAs, and perform other functions. Forinstance, a DVAES-enabled device could be a part of common network andtelecommunications devices such as VoIP telephony adapters, cable andDSL modems and routers, integrated access devices, fixed-line andwireless devices, dedicated network appliances, VoIP telephones,residential gateways, set top boxes, cellular telephones, automotivetelematic devices, wearable computing devices, media center controllers,mobile computing devices (e.g. PDAs), or any other device which hasnetwork access.

Most existing customer premise equipment and consumer devices that couldbe DVAES-capable lack the DVAES components, configurations, andmembership credentials that would allow them to participate in one ormore DVAESs. Adding a DVAES-enablement layer of software components andconfiguration materials would make them DVAES-enabled, and thus capableof participating within the DVAES architecture. In many instances, itwould probably not be feasible to retrofit existing devices so that theyare DVAES-enabled, although this is certainly possible. However, itshould be no problem to add the DVAES-enabling elements to manydifferent types of commonly sold and distributed customer premiseequipment and/or consumer electronics. Thus, the Inventors envision thatin the future, many existing types of commonly sold and distributedcustomer premise equipment and/or consumer electronics could be sold,pre-configured, with all the elements necessary for the device to beintegrated into a DVAES. This would provide an easy way for customers toactivate a DVAES service using equipment that is already present intheir home.

It is appreciated by those skilled in the art that DVAES-enabled devicesmay be virtualized.

Furthermore, the DVAES architecture components may integrate with andsupport components of extant legacy systems to facilitate migration fromcentralized voice service platforms to a distributed architecture.

FIG. 6 shows elements of a typical piece of DVAES-enabled equipment thatwould be located at a user's home or office. Alternatively, as notedabove, the DVAES-enabled equipment could be part of a mobile computingdevice which connects to a wireless network. The DVAES-enabled equipment3000 includes a hardware layer 3100, an operating system (OS) layer3200, an Audio and Signaling Layer 3300, and a DVAES-enablement layer3400. The DVAES-enablement layer includes DVAES-enabling components,including at least one instance of a VAA 3410, 3410 b, and VAAconfiguration materials 3420.

A DVAES-enabled hardware layer 3100 preferably provides computerhardware and firmware for the operation of DVAES architecturecomponents. The hardware and firmware of the DVAES-enabled hardwarelayer 3100 is used to provide an operable computing platform upon whichto host the DVAES-enablement layer 3400. The DVAES-enabled hardwarelayer supporting the DVAES-enablement layer described herein isexemplary in nature and may be provided by any hardware, firmware, orsoftware combination that fulfills the operating requirements of theDVAES-enablement layer.

A DVAES-enabled device's hardware layer 3100 comprises an operationalcomputing platform combined with specialty hardware interfaces andfirmware to enable the DVAES architecture. A DVAES-enabled device'shardware layer 3100 comprises a processor 3112; volatile 3114 andnon-volatile memory 3116 (e.g. RAM, ROM, FLASH or other forms of memorydevices); a bus 3118; one or more optional hard disks or other massstorage devices 3119, optional I/O interfaces 3113 which could includeUSB, serial, parallel and other types of interfaces; optional audio I/Odevices which would typically include a speaker and a microphone 3140;optional telephony interfaces 3110; at least one network interface 3120;and optional DSP/audio hardware/drivers 3130.

A DVAES-enabled device requires a sufficient processor to effectivelyrun the DVAES components assigned to the device. Some aspects ofhardware may be provided using software or firmware; this is animplementation decision that may require additional CPU processingcapability or may require the use of clustering technologies. An exampleCPU is an Intel XScale processor from Intel Corporation of Santa Clara,Calif.

A DVAES-enabled device requires sufficient RAM to effectively run theDVAES components assigned to the device. The amount of RAM required isimplementation dependent. Additional RAM can improve the effectivenessof a DVAES-enabled device and optionally may be included. For instance,adding additional RAM may enable the device to perform more complexvoice applications A DVAES-enabled device requires sufficient persistentstorage such as ROM, FLASH memory (EEROM or other suitabletechnologies), or other types of non-volatile memory to persistentlystore information required for the operation of the devices within thedevice itself. Examples of types of information that may be stored inthe non-volatile memory include device firmware, configurationmaterials, copies of an operating system, VAA components, VAAconfigurations, VA components, and user personalization information.Persistent storage such as FLASH memory may be provided within theDVAES-enabled device, or may be alternatively accessed using a wired orwireless I/O interface.

A DVAES-enabled device may be equipped with a local hard disk for thepersistent storage of materials as described above.

Optional I/O interfaces, including USB, serial, and parallel interfacesmay be provided for extensibility, and to interconnect with externaldevices and accessories. Example accessories might include a wirelessnetworking interface or a USB-based Bluetooth wireless connection for aheadset.

Optional audio I/O Devices such as one or more speakers and a microphonecould be used to interact with the user. The speaker(s) would play audioto the user, and the microphone would pickup the user's spokenresponses.

At least one network interface would be used to connect theDVAES-enabled device to the network. The network interfaces could be awired interface, a wireless interface, an infrared interface, or anyother technology that would enable the DVAES-enabled device tocommunicate over the network. Preferably the network interface wouldprovide 10baseT or better Ethernet, or 802.11-b/-g wirelessconnectivity.

The device could also include optional telephony interfaces, such as FXOand FSO interfaces for connecting with telephony devices and centraloffice hardware. These interfaces may be used to connect theDVAES-enabled device to a PSTN, VoIP, or an internal PBX system.

An optional DSP may be included to manage audio capture/playback,digitization, and speech conversion. An example DSP is a FreescaleStarCore, from Freescale Semiconductor of Austin, Tex.

The device might also include other optional hardware, such asinterfaces for a display screen, a keyboard, a mouse, a touch screen,other pointing device and the like (all not shown) as appropriate forspecific implementations of a DVAES-enabled device.

A DVAES-enabled device preferably provides an operating system layer3200 to abstract the functionality of the hardware and network voice anddata implementation details from the functionality of theDVAES-enablement layer 3400. The OS layer 3200 comprises traditionaloperating systems and device drivers, as well as traditional networkingapplications such as the network protocol stack, routing, firewall, andrelated service applications. The operating system layer supporting theDVAES-enablement layer described herein is exemplary in nature and maybe provided by any operating system that fulfills the operatingrequirements of the DVAES-enablement layer.

The Operating System layer of a DVAES-enabled device comprises anoptional cache 3210, network components 3220, operating Systemcomponents 3230, configuration materials 3240 and an audio and signalinglayer 3300. An optional web interface (not shown) for configuring deviceconfiguration materials may also be included.

The DVAES-enabled device's operating system could be based on anembedded version of the Linux or VxWorks operating systems, althoughembodiments that support embedded versions of Microsoft Windows (andWindows for Devices), Symbian, QNX, PalmOS, BlackBerry OS and otherembedded operating systems are also envisioned. The DVAES-enableddevice's operating system additionally comprises the drivers, modules orloadable operating system components required to identify, interface to,and operate each type of hardware present in a DVAES-enabled device. InLinux-based operating systems, these drivers are sometimes calledmodules or device drivers. Modules for FXS and FXO interface cards, aswell as audio I/O devices are envisioned as part of the device operatingsystem layer.

The network components of a DVAES-enabled device include protocolstacks, network management components (e.g. SNMP management components),routing software, protocol and network address translation components,and firewall components. These components include any requiredconfiguration files or other software required to make them operate. Thenetworking components preferably operate using the TCP/IP protocol,although other protocols are envisioned.

In some embodiments, the DVAES-enabled device's networking layer mayinclude networking components for managing VoIP calls, including, forexample, such components as a SIP stack or H.323 services. Componentsrequired to interface to external PSTN and PBX systems are supported aspart of this layer. Collectively, all such components are consideredpart of the network layer of a DVAES-enabled device.

The network management components may include remote management andmonitoring agents configured to manage and report upon the status ofDVAES-enabled device operations to one or more DVAMSs. The networkmanagement components may include SNMP-based trap mechanisms, SNMPagents, TR-069 agents, or other remote management and monitoringcomponents of types well known to those skilled in the art ofnetwork-based device management and monitoring.

A DVAES-enabled device preferably provides an audio and signaling layer3300 to abstract the functionality of network voice and data mechanismsfrom the OS layer 3200 and from the DVAES-enablement layer 3400. Theaudio and signaling layer 3300 provides audio I/O device abstraction forDVAES components. In one exemplary embodiment, the audio and signalinglayer 3300 abstracts a local audio device such as a microphone andspeaker into an input/output device associated with a VAA, withoutregard to the type or location of the physical audio I/O device. Inother embodiments, the audio and signaling layer 3300 provides supportfor network voice components of the DVAES-enabled device, such assupport for VoIP.

The audio and signaling layer 3300 of a DVAES-enabled device providesDVAES component interfaces to hardware audio and signaling services andnetwork audio and signaling services (collectively audio and signalingservices). Hardware audio and signaling services includes FSX interfacesconnected to a telephone and audio I/O device (speaker/microphone or USBheadset) interfaces. Network Audio & Signaling Services includes supportfor Voice Over network Signaling and transport protocols and standardssuch as SIP, H323, MGCP, RTP, PSTN and WiFi or Bluetooth based audio I/Odevice interfaces. The Audio and Signaling Layer applies aspects ofdevice hardware, the supporting operating system and drivers, andconfiguration materials for establishing sessions between audio andsignaling services and one or more DVAES components. It maps the rulesfor establishing a connection between audio and signaling services toone or more DVAES components (e.g a VAA). Optionally, the Audio andSignaling layer may provide such mapping based on configurable rules.The applications of mapping rules may be based on physical and derivedattributes of audio and signaling services.

An audio and signaling session is a uni-directional or bi-directionalconnection established with hardware audio and signaling source(s) or aconnection established with a network audio and signaling source. Theaudio and signaling layer 3300 manages audio and signaling sessions byproviding a connection between DVAES components (e.g. a VAA) and theaudio and signaling services. As an example, when a user activates aphone connected to an FXS interface of the DVAES device, the audio andsignaling layer 3300 creates an audio and signaling session that is madeavailable to a VAA based on mappings and rules. Generally, audio andsignaling sessions are connected with one or more voice sessions in theline manager component of a VAA, as will be explained in more detailbelow. The features of the audio and signaling layer vary in accordancewith the capabilities of the underlying hardware. These features mayfurther include aspects of hardware device controllers, such as homeautomation controllers, enabling voice applications to control locallyconnected hardware and control systems. Furthermore, the audio andsignaling layer can support one or more audio and signaling sessionsdepending upon capabilities of the hardware, software resources andconfiguration of the DVAES-enabled device.

Thus, by utilizing the audio and signaling layer 3300, a DVAES-enableddevice can utilize audio associated with standard analog telephones, amicrophone/speaker combination (such as a microphone and speaker),VoIP-based telephony devices, locally connected telephones (e.g. FSXinterface connected telephone devices), and other devices located on aPSTN or data network.

Thus far, the discussion has not provided any detail about the DVAESenablement layer 3400. Before describing the details of the DVAESenablement layer, a description of how a particular piece ofDVAES-equipment is configured and registered will be provided. Therewill also be a discussion about updating or replacing configurationmaterials in a DVAES-enabled device. After that, we will return to adetailed description of the DVAES enablement layer of a DVAES-enableddevice.

A DVAES-enabled device optionally supports device-level configurationmaterials that define the configuration and operation of the hardware,operating system, and networking, and audio and signaling layers of thedevice. The configuration materials may be stored in any combination offlat files, XML files, registry databases, or other databases asdictated by the operating system, network, and audio and signaling layercomponents being configured. In some embodiments, the configurationmaterials may be stored in a cache.

In a first embodiment, the device configuration materials are notoriginally stored in non-volatile memory of the DVAES-enabled deviceitself. Rather, these materials are loaded into each DVAES-enableddevice when it boots and requests its network address and otherinformation from the network. Traditional mechanisms for providingdevice configuration materials include the well-known BOOTP, DHCP, andthe TFTP protocols. Alternatively, some or all of the deviceconfiguration materials may be stored within a network-based cachemechanism such as a content distribution service, and may be obtained bythe device using protocols appropriate to accessing the contentdistribution service. Some of these protocols include HTTP and FTP. Inother embodiments, at least some of the device configuration materialsare stored on each DVAES-enabled device, either within the DVAES-enableddevice's cache, or in a separate configuration area used to persistentlystore these materials. Examples of such persistent storage areas includethe FLASH memory or optional hard disk drive of the DVAES-enableddevice.

In a first example embodiment, device configuration materials stored ona DVAES-enabled device include parameters and configurations thatcontrol how the modules and services of the operating system and networklayer of the DVAES-enabled device operate. Specific examples of thesemodules and services include device drivers and loadable modulemanagement (e.g. enabled/disabled, specific mappings), networkingparameters, (including such items as network boot definitions), PPOEcredentials, routing parameters, NAT configurations, and firewallsettings.

Additionally, some embodiments of the device configuration materialsdefine telephony line interface definitions, including line interfaceconfiguration parameters, associations between specific line interfacesand operating system device drivers, and the like.

Other exemplary embodiments of the device configuration materials definethe use of local audio interfaces such as a local microphone andspeaker, and provide configuration information that map these devices toaudio and signaling layer software that provides a telephony callsession that is usable by a VAA (described below). Alternate embodimentscombine loadable module management with specific plug-in audio devices,such as USB wireless (Bluetooth) interfaces that support the use ofBluetooth-enabled local audio devices such as a wireless headset.

In some particular embodiments, a DVAES-enabled device may store withinits configuration materials a list of its capabilities, including suchitems as the amount of memory, processor resources, telephony lineinterfaces, components loaded and configured for use, etc. Thisinformation may be automatically generated on demand, may be static, ormay be a combination of static and dynamically generated information.The capabilities information provides the DVAES-enablement layer andDVAES management systems (such as DVAMS) information about thecapabilities of each specific device.

In still other embodiments, the device configuration materials definespecific limits and capabilities of a DVAES-enabled device, and mayinclude limits upon the capabilities of the device. These deviceconfiguration materials may define artificial limits in the capabilitiesof a device, either for enabling resource sharing between components orto provide software limitations to capabilities that may be removed whenthe device is activated. For example, specific capabilities of aDVAES-enabled device may only be enabled based upon the level of servicethat a user purchases. In some embodiments, where clustering orcooperative processing of DVAES-enabled devices is a factor, theconfiguration materials also provide details on how the DVAES-enableddevice interacts with other DVAES-enabled devices in the cluster.Clustering is described in more detail below.

The DVAES-enabled device's configuration materials also define basicconfigurations of DVAES-enabled components present on a DVAES-enableddevice. In addition, device configuration materials define the order andparameters that indicate how these components are started when aDVAES-enabled device is started or restarted. The services managed bythese configuration materials include operating system, local hardware,networking, interface components, and DVAES-enablement layer componentssuch as VAAs. The configuration materials also define the manner inwhich each DVAES-enabled device is initially configured for use on thenetwork and with a DVAES.

Device configuration materials may define the locations and addresses ofwell known services that are available to the device, such as a DVAMSwith which a device is to register with, or the location and addressesof specific network services usable by the device such as known CDSservers, external ASR or TTS servers, and the like.

In addition to definitions for external DVAES services, deviceconfiguration materials also define the DVAES-enabled services such asthe number and configuration of VAAs operating on a DVAES-enableddevice, locally cached copies of DVAES-specific device registrationmaterials, and configuration materials that associate specific hardwareservices and devices with each local VAA instance.

In some more specific embodiments, device configuration materialsdescribe specific addresses for each DVAMS the device must reportoperational details to (for example, using SNMP).

In some embodiments, the device configuration materials may provide, ormay reference, specific device identification materials that may be usedto uniquely identify the specific DVAES-enabled device. These materialsmay include unique MAC addresses for use by the network interfaces, aunique device ID assigned to the device, a public-private key pair, orother materials. In some embodiments, the configuration materials mayinclude authorization and authentication materials that permit access toexternal services. These authorization and authentication materials maybe unique to the DVAES-enabled device, or may be shared across aplurality of devices.

In some embodiments, the device configuration materials described abovemay be pre-loaded onto a DVAES-enabled device before it is shipped fromthe manufacturer. In other instances, the configuration materials may bechanged as part of a registration process or other automatic process, ormay be configured manually using a user interface such as a web-basedinterface of the type commonly provided with network-based customerpremise equipment. The configuration materials are preferably stored innon-volatile memory (e.g. FLASH or ROM) or other persistent storage suchas a hard disk and may be protected against tampering using well-knowncryptographic techniques such as MD5 or SHA-1 hashes.

External DVAES components such as a DVAMS may make changes in theconfiguration materials of a DVAES-enabled device. These changes may bemade by providing replacement configuration materials for theDVAES-enabled device, either by directly updating the existingconfigurations, or by other means well known to those skilled in theart. In some particular embodiments, a management protocol that iseffective to set specific device configuration materials and theirvalues may be implemented as a service on a device.

Each DVAES-enabled device is registered with at least one DVAMS. Groupsof DVAES-enabled devices may also be registered. Registration is theprocess by which a DVAES-enabled device becomes part of a DVAES.

A DVAES-enabled device is registered with at least one DVAMS when it isfirst integrated as part of a DVAES. Registration is preferablyautomatic and performed at the hardware and firmware level. In oneexample embodiment, the registration process is started when aDVAES-enabled device performs a broadcast to locate a DVAMS, receives aresponse that includes information about the DVAMS's address, and thenproceeds with the registration as described below. In some embodiments,the broadcast/response is part of a DHCP or BOOTP request/response cyclethat occurs when the DVAES-enabled device is connected to a network. Inalternative embodiments, the broadcast/response mechanism is performedafter the DVAES-enabled device is present on the network (e.g. has anetwork address). Alternatively, if a DVAES-enabled device is alreadypresent on a network (e.g. has a network address), a DVAES-enableddevice may start the registration process by contacting a well-knownservice such as a directory or registration service from which thelocation of an appropriate DVAMS service may be determined.

Once the address of a DVAMS is established by a DVAES-enabled device,the DVAES-enabled device connects to the registration service publishedby the DVAMS and registers itself. Preferably, the DVAES-enabled deviceprovides a unique machine ID along with owner, location, hardwareconfiguration, software configuration, and available user, group ofusers, VAAs, and VA information as part of the registration process.

Authentication optionally may be required in order to permitregistration. Authentication techniques are well understood to thoseskilled in the art and may include hardware certificates,account/passwords, or other mechanisms. Once satisfied that the requestis both valid and should be completed, the DVAMS updates itsregistration directory with the information provided by theDVAES-enabled device, and enables the DVAES-enabled device for use.Copies of DVAMS registration materials are preferably stored in theconfiguration materials associated with each DVAES-enabled device, asdescribed above.

Each DVAES-enabled device may be registered with one or more DVAMS. In afirst example embodiment, each DVAES-enabled device is registered with aDVAMS associated with a specific DVAES. In other embodiments, wherethere is a plurality of DVAES implementations (for example, where twodifferent vendors have deployed the DVAES architecture), a DVAES-enableddevice may register with a DVAMS associated with each DVAES. In thissecond example, a plurality of DVAMS registration materials may bestored as part of the configuration materials described below.

FIG. 7 illustrates an exemplary set of process steps taken by a devicein order to register itself with DVAMS. In step 4110, a device initiatesa connection to a DVAMS. The connection can be made using informationthat is pre-configured or configured on the device and that specifiesthe DVAMS to access. Alternatively, the connection can be made byreferencing a service that specifies the DVAMS to access. In otherembodiments, the device may download information that specifies theDVAMS to access, or use downloaded information that specifies aparticular service provider, and the service provider may then specifythat the device register with a particular DVAMS.

In step 4120, the DVAES-enabled device then provides information aboutthe device to the DVAMS, including its unique ID, any DVAMS orDVAES-credentials already in the device, and possibly a capabilitieslist. Unique IDs may take the form of MAC, digital certificate, publickey, or any other method for uniquely identifying the individual device,and are of well-known construction to those skilled in the art.Optionally, credentials authorizing the registration of the device withthe DVAMS are passed. These credentials may take the form of an accountID/password, a digital certificate, or any other authorization materialsthat may be used by the DVAMS to determine that the device is authorizedto be registered. A capabilities list would describe the capabilities ofthe device, including line interfaces, audio interfaces, networkinterfaces, available memory, and any other pertinent items. Thecapabilities list includes a DVAES-enabled components list andconfigurations.

In step 4130, the device receives a membership credential from theDVAMS. The membership credential may be directly downloaded to thedevice, or it may be made available in a directory, database, or contentdelivery service, from where it is subsequently distributed to thedevice. Each membership credential associates a specific device, bymeans of its unique ID, with a specific DVAMS.

In step 4140, the device stores the membership credential within thedevice. Preferably, the DVAES membership credential is stored with thedevice configuration materials, but alternatively may be stored withspecific VAA configuration materials or may be stored in other placeswithin the device. In some embodiments, the membership credential isfurther stored in a content delivery service.

Finally, in step 4150, the device downloads (or has distributed to it)and stores any required DVAES-enablement layer components andconfigurations from the DVAMS. In some embodiments, the device receivesthe actual required components and configurations from a DVAMS. In otherembodiments, the device receives a list of components that are requiredby the device, and the device is responsible for obtaining thesematerials via alternate means, such as accessing the materials from aVASS or a CDS.

The DVAES enablement layer materials may include various implementationsof one or more VAAs, VAA configuration materials, device configurationmaterials, user configuration materials, device components, and anyother required components of the DVAES architecture. Each implementationof downloaded materials may be embodied as different instances orversions that may have similar or differing features and capabilities.

Embodiments of DVAES components may be developed using various differenttechnologies that operate on differing underlying platforms. In someembodiments, the DVAES components may be implemented using Java, C,assembly language, C++, C#, or other programming languages that areadvantageous for the particular application environment of the device.

In some embodiments, DVAES components may be programmed as stand-aloneprograms, as daemons or services, or as executable object or classcomponents as are applicable to the deployment environment. Whenoperating as a service or daemon, each DVAES component preferablymonitors its configuration and if changes occur in the underlyingconfiguration components, each DVAES component reloads the newestconfiguration, and optionally restarts itself to make the newconfigurations effective. In other cases, an external service monitorsthe changes and notifies the applications that are affected to restartor reload their configuration files.

DVAES components are, in most embodiments, inherently downloadable intoa DVAES device. DVAES components may be directly downloaded, or ifstored in a non-volatile cache, automatically updated by refreshingtheir cache-based storage. The downloadable nature of the components isembodied in their packaging method. For example, in some embodiments,the DVAES components may be written in Java and are deployed in Jar orEar files. In other deployment environments, DVAES components may bewritten in C++ or C# and are deployed as part of .NET assemblies using.MSI install packages. In other embodiments, the applications may bedeveloped in C or assembly language, and require third party installers.

The above registration process is exemplary in nature. Many othervariations would be possible to register a piece of DVAES-enabledequipment with a DVAMS.

FIG. 8 illustrates another process embodying the invention, by which aDVAES-capable device is registered, has all required componentsdownloaded into it and is made ready for use by one or more users.

In step S110, the device starts up. In step S120, the device checks itsstate to determine if it is already registered with a DVAMS.

If the device is unregistered, in step S122 the device collectsinformation from the user (account number, etc) and starts the combinedregistration process. In step S124, the device registers itself inaccordance with a “device registration process,” such as the one justdescribed in connection with FIG. 7 . In step S126, the device wouldreceive a membership credential.

If the device determined, in step S120, that it was already registered,the method would have proceeded directly to step S130. If the result ofthe check in step S120 indicated that the device was unregistered, thenafter the registration process is complete, the method will eventuallyreach step S130. In step S130, the device performs any requireddownloads and updates its configuration materials. The device alsore-starts any required services, or optionally reboots.

In step S140, the device provides user-provided account materials, andthen in step S150, the device associates itself with one or more usersof the system and reports that association to a DVAMS.

In step S160, the device performs any required downloads and updates itsconfiguration materials. Also in step S160, the device would re-startany required services, or optionally reboots, depending on whetherupdated materials requiring such an action were downloaded. In stepS170, the device registers any required VAAs with a DVAMS. In step S180,the device receives VAA membership credentials. Finally, in step S190,the device performs any required downloads and updates its configurationmaterials, and the device re-starts any required services, or optionallyreboots.

The above discussed methods explain how a DVAES device would initiallyregister itself with a DVAMS, and then acquire any necessaryconfiguration materials and/or credentials. For various reasons, a DVAESdevice may need to replace or update its configuration materials afterthe initial configuration process. For instance, a DVAMS may identifythat one or more components on a device are missing, or are out of dateand need to be replaced. An example of components to be downloaded orupdated would include a VAA, configuration materials related to a VAA,device drivers, program executables, device configurations, and thelike. Generally, replacement of these components in the device may notrequire a complete reset of the device and can be effected bydownloading the components, and performing a restart of the affectedservice or services. In some cases, rebooting the device may occur tore-start the affected service or services. The decision to restartservices or reboot the device is implementation dependent.

In other embodiments, a DVAMS may generate new deploymentspecifications. This occurs when the allocation of specific users, VAs,and VAAs to specific devices are changed in the DVAMS. These deploymentspecifications are used by the DVAMS to produce updated configurationmaterials, and may further introduce requirements for other, new,additional, or updated components into a device.

The distribution of new or updated materials may be performed in one ofseveral ways. In a first embodiment, the materials may be downloadedusing a direct download instigated by a DVAMS. Alternatively, they maybe distributed using a pull-type distribution mechanism using a serviceinterface, a web service, or the cache. An alternate mechanism foridentifying and distributing materials into a device is to have aservice that periodically checks the versions of all DVAES-enableddevice components to determine if new components are needed. Thisservice would download new components to a target when they areidentified.

Having described how a DVAES-enabled device is first registered andconfigured, we will now turn back to a discussion of the DVAESenablement layer of a typical DVAES-enabled device. References will bemade to FIGS. 6 and 9 , which illustrates various components of theDVAES enablement layer.

The DVAES-enablement layer 3400 includes one or more VAAs configured touse features of the DVAES-enabled device and operable to perform VoiceApplications (VAs). The appropriate VAAs and VA components may bedeployed or loaded into each DVAES-enabled device as managed by a DVAMS,as described above. The VAAs and VA components are configured usingappropriate configuration materials and enable the performance of VAcomponents by a DVAES-enabled device. More details about how the DVAMSmanages the VAAs to perform VA components, and how VAs are rendered tothe device will be provided in other sections below.

The DVAES enabling component layer 3400 of a DVAES-enabled devicecomprises VAA configuration materials 3420 and at least one VAA. TheDVAES-enabling layer of devices provides support for performing voiceapplications, enabling these devices to be effective in providing theperformance of distributed voice applications in a widely distributedarchitecture.

As noted above, the concept of a VAA is partly an abstract logicalconstruct. In the end, a VAA must operate through some physical device.But VAAs need not be tied to a single device. A VAA may be deployed as asingle, integrated service, or as a collection of individual servicesthat cooperatively operate to provide VAA services to a DVAES. Thecomponents that make up a VAA may be pre-distributed or pre-loaded ontoa device upon manufacturing the device. More typically, however, VAAswould be deployed on an as-required basis into DVAES-enabled devices,including standard consumer electronics and networking devices, toenable the performance of voice applications by these devices.

Also, in some embodiments, a plurality of VAAs may be present on asingle DVAES-enabled device. These VAAs may perform VA components thatmay require a specific instance of a VAA. In other embodiments, a devicemay have a plurality of VAAs, each associated with a different DVAES.The different VAAs may all be associated with a single user, or thevarious VAAs may be associated with different users. Some examples ofhow these variations could arise are given immediately below.

In one instance, a single user may end up with multiple VAAs loaded onhis local device. In this instance, a first VAA could be configured toperform VA components related to the user's personal use, and a secondVAA could be configured to perform VA components related to the user'sprofessional use.

In another similar situation, the user may have registered for voiceservices with two separate service providers. For instance, assume thata first DVAES is operated by a first operator such as Comcast, and asecond DVAES is operated by a second operator such as Verizon. Furtherassume that the user has registered to obtain voice services from bothComcast and Verizon. Here again, Comcast could be providing the userwith services related to his personal life, and Verizon could beproviding services related to the user's professional life. In anyevent, two separate VAAs may be loaded onto the user's local device, oneof which is registered with and controlled by the Comcast DVAES, and theother of which is registered with and controlled by Verizon. There wouldbe no inherent conflict in loading both VAAs onto the same customerdevice. And the DVAMS for the respective service providers would eachcontrol, update and maintain their respective VAAs loaded onto theuser's device.

In still other embodiments, a single piece of local equipment may beproviding support for a plurality of users. In this instance, each ofthe users could make use of a different VAA. Here again, the VAAs couldbe provided by, updated and maintained by the same DVAES, or bydifferent DVAESs operated by different operators.

There are a plurality of reasons for deploying individual devices thatutilize disparate VAAs, including, for example, different authorizationand personalization domains, separation of content and users, anddisparate DVAES. Different operator, authorization, and personalizationdomains occur when a DVAES operator or operators of external systemswith which VAs interface require separation of authorization andauthentication materials. In some cases, these materials are mutuallyexclusive or would require additional logic to determine which set touse from a plurality of options. Use of a plurality of VAAs, eachoperating within disparate operator, authorization, or personalizationdomains, overcomes this obstacle without introducing run-timecomplexity.

In other instances, a VAA could operate in a virtualized manner, and notbe bound to specific hardware until they are executed. One example ofthis type of virtualization is deployment of a VAA using software suchas VMWare (commercially available from EMC, of Hopkinton, Mass.), Xen(public domain), or virtual server (commercially available fromMicrosoft of Redmond, Wash.). This means that a VAA may be loaded onto aparticular DVAES-enabled device only after a user identifies that hewishes to use that device to access a voice application.

Each VAA is configurable using VAA-specific configuration materials.These materials may be stored within a DVAES-enabled device, or may beloaded from cache, a CDS, or from a DVAMS or VASS on an as-needed basis.

In one aspect, the VAA configuration materials comprise definitions ofthe services and components that the VAA requires to operate asconfigured. These services and component definitions may name serviceslocal to the device (such as an ASR or TTS service), or may name remoteservices (such as remote ASR or TTS services, a DVAMS, TASS, or CDSservice). The configuration materials may also identify the number ofinstances of each service to start for the voice browser pool (e.g. anumber and types of voice browser sessions required), and may furtherspecify the voice applications that each voice browser instance mayperform. Default voice applications, e.g. voice applications that areassociated with a specific voice browser instance on startup, also maybe assigned to each voice browser instance.

The VAA configuration materials also provide configuration settings fordetermining the items to log, log levels, and locations where logsshould be sent (e.g. the DVAMS). The items to log may be specified on aservice or component level, and may be detailed down to the logging ofthe performance of specific VA components.

The association between Voice Browsers and specific voice sessions maybe made on a static (configuration) basis as defined in the VAAconfiguration information, or the association may be made on anon-demand basis. Specifically, each voice browser instance may beassociated with zero or more voice sessions.

Within each VAA's configuration materials are further configurationmaterials for the configuration of specific voice browser instances andcomponents. Each set of voice browser configuration materials permitsmonitoring of each voice browser and internal component states,including specific items such as Voice browser start, Voice browserstop, Voice browser errors, Voice browser VA component currentlyprocessing, and Voice browser cache state changes. The Voice Browser isalso configurable to monitor and log VA states, including: Initial URL,Page Transitions, Log tags, Event Catch tags, Session variables, VAcomponent Errors, Input Fields, and specific prompts.

In addition to other things, the VAA Configuration materials may includespecifications for the following items:

-   -   Services and Number of instances of each service;    -   Associations between services;    -   Optional Associations of OA&M Service to VAA Services;    -   Line Manager Configuration Items;    -   Number of Voice Sessions and Types;    -   Association of Voice Sessions to Audio and Signaling sessions;    -   Association of Voice Sessions to ASR sessions;    -   Association of Voice Sessions to TTS sessions;    -   Association of Voice Sessions to Voice Browser Sessions;    -   Optional Associations of Voice Sessions to OA&M Services;    -   Voice Browser Configuration Items;    -   Number of Voice Browser Sessions and Types;    -   Association of Voice Browser Sessions to Voice Sessions;    -   Association of Voice Browser Sessions to ASR sessions;    -   Association of Voice Browser Sessions to TTS sessions;    -   Associations of Voice Browser Sessions to an OA&M Service;    -   ASR Service Configuration Items;    -   Association of ASR Sessions to Voice sessions;    -   Associations ASR Sessions to OA&M Service;    -   TTS Service Configuration Items;    -   Number of TTS Sessions and Types;    -   TTS number of Voice Browser Sessions and Type;    -   Association of TTS Sessions to Voice sessions; and    -   TTS associations of OA&M Service;

The configuration materials may also specify the amount of cache usedand caching rules. Optionally, these cache configuration rules may bespecified by the DVAES-enabled device configuration.

An exemplary VAA will now be described with reference to FIG. 9 . FIG. 9shows an exemplary voice application agent (VAA) 6000 comprising a linemanager 6110, an optional cache 6200, a pool of running Voice Browserinstances 6120 a, 6120 b, 6120 c, a Local Service Engine (LSE) 6130, anAdvanced Speech Recognition (ASR) Service 6140, a Text-to-Speech (TTS)Service 6150, and an Operations, Administration, and Monitoring (OA&M)Service (6160). The VAA cache 6200 may be broken down into an optionalcache interface 6210, an optional Cache 6230, and an optional cachemanager service 6220.

The line manager 6110 provides comprehensive voice session managementfeatures to a VAA. A Voice Session is a bi-directional, managedconnection between a voice browser and an Audio and Signaling session. Aline manager has channels. Channels could operate in Basic and/orTelephony Mode.

A basic channel provides a uni-directional or bi-directional interfaceto an Audio and Signaling Session that supports, for instance, amicrophone & speaker audio source. This implementation typically relieson the drivers of the DVAES enabled device. A telephony channel is amore advanced implementation of a channel as it provides an interface toAudio and Signaling Sessions that support phone type connections (e.g.,an analog telephone, a cordless telephone, WiFi, VoIP, Skype, etc.). Atelephony channel propagates audio and signaling eventsuni-directionally or bi-directionally between a Voice Browser to anAudio and Signaling Session. The Line Manager can support multiple VoiceSessions based on the number of audio and signaling sessions supportedby the DVAES enabled device.

The line manager component 6110 manages instances of Voice Browsers. Theline manager may create voice browser instances when a VAA is started.One or more voice browsers are managed in a pool by the line manager.The specifics of the number and type of voice browsers that areactivated are based on VAA configuration data.

When an audio and signaling session is initiated, a request is made bythe audio and signaling layer 3300 to the VAA's line manager 6110 for avoice session. The line manager establishes a voice session by acceptingthe request, and associates the voice session with one or more VAAcomponents based on VAA configuration data or rules. In someembodiments, this assignment activates a voice browser. In some cases,the line manager instantiates new Voice Browser instances if asufficient number of Voice Browser instances are not available. The linemanager 6110 also manages the starting, stopping, andsuspension/resumption of specific instances of voice browser sessionsbased on VAA requirements, VA requirements, system, voice browser, audioand signaling session instructions, and/or the configuration materials.

Upon activation of a voice browser, the line manager 6110 provides thenewly activated voice browser data provided by an Audio and SignalingSession. In the case of a session connected to a public telephonysystem, the data provided may include any call information provided bythe public telephony system such as call time, caller ID, and callername.

An Audio and Signaling Session connected to a voice session could be asbasic as an analog phone connected to an FXS or FXO interface on a DVAESenabled device, or as advanced as a VoIP or Skype-like connection. TheAudio and Signaling Session could also be a PBX that treats the VAA asan extension to the PBX. A telephony channel in the line manager may beactivated instantly when the user picks up a telephone handset connectedto an FXS interface of a DVAES enabled device, or it could be activatedwhen a PBX sends a request to a DVAES-enabled device to accept a SIPcall. Effectively, in doing so, the line manager enables the VAA toperform a Voice Application on the off hook event of a connectedtelephone device, and/or when a DVAES enabled device receives a phonecall.

During the voice session, a telephony channel accepts and propagatesstandard telecom instructions and call/network data (e.g. ANI) to andfrom the Audio and Signaling Session. Examples of such instructionsinclude “off Hook”, “Dial,” “Bridge,” “Transfer” etc.

The Line Manager, based on rules or system events, may switch a voicesession connection with an Audio and Signaling Session from a firstvoice browser to a second voice browser. This has the effect ofswitching the user from one VA to another. Alternatively, the LineManager may accept instructions from a voice browser to pause a voicesession and switch the voice session to an alternate voice browser. Asan example, the Line Manager could permit the user to switch voicebrowsers and launch a new Voice Application based on a “Hot word” voicecommand. So, in this instance, as soon as the voice browser determinesthat a hot word has been spoken, the voice browser would make therequest to pause the voice session and to switch the voice session to analternate browser. In still other instances, when a telephone channel isbeing used, the switch to a different or new voice browser might betriggered by keying a specific DTMF key sequence. Whenever such aninstruction is received by the line manager, the current voice browseris paused and the voice session is connected to a new voice browser.

FIG. 19 is intended to illustrate some exemplary uses of the linemanager to connect a plurality of audio and signaling sessions (F310 a,F310 b, F310 c) to one or more voice browsers (F210 a, F210 b, F210 c),using voice sessions.

In a first example, A&S sessions F310 a and F310 b are jointly connectedwith a single voice session, F110 a. The voice session F110 a isconnected to one or two voice browsers F210 a, F210 b sequentially or inparallel, depending upon the voice application being performed. This hasthe effect of providing voice application(s) to a plurality ofsimultaneous users (similar to a conference call during which a VAperforms).

In a second example, A&S session F310 c is connected to a voice session(F110 c), which is further connected to a single voice browser (F210 c).

In a third example, A&S Session F310 c is connected to a plurality ofvoice sessions F110 b, F110 c, which are in turn associated with aplurality of voice browsers F110 a, F110 b, F110 c. An example of thistype of configuration might include a user participating with one ormore voice browsers, and a second VA providing “hot word” recognitionand processing. The line manager 6110 provides logs detailing linemanager events to the OA&M Service 6160. The Line Manager Log items mayinclude details of one or more voice sessions, including Start time, EndTime, Voice Browser ID, Line Manager Channel ID and Type, Audio andSignaling session ID and Type. Additionally line manager log informationmay include detailed error messages. The logging level provided by theline manager is configurable and provides varying levels of logging inresponse to a configuration setting.

The Line Manager may also provide error notifications to the VAA OA&MService. The error notifications may range from severe to warning, andthe detail level could be configurable.

As noted above, the VAA additionally may include a cache subsystem 6200,that itself may include a cache interface 6210, a cache 6230, and anoptional cache manager 6220 that provides transparent local caching ofVA components within the context of each VAA. In some embodiments, thecache is shared between a plurality of VAAs on a specific DVAES-enableddevice. In such a configuration, each voice browser could make requestsfor VA components to the local caching subsystem via the cache interface6210. Thus, if the requested component is located in the cache, therequested VA components could be fetched from the cache 6230, instead ofmaking a network request for the content.

In embodiments that include a plurality of caches (e.g. a VAA and adevice cache), the VAA cache may make a request to the devices cache,which makes a request over the network to the source. The configurationof cache behaviors is defined by caching rules. Unlike traditionalproxy-based cache systems, the cache with a cache manager componentextends local as-needed caching algorithms to content distributionservice components, and further provides predictive and push-based cacheupdates to the proxy server. The size of the cache, cache updatefrequency, caching rules, caching schemes to use, lookup locations for aCDS, and content sources at a DVAMS and/or a VASS are specified as partof the appropriate layer's (e.g. device's, or VAA's) configurationinformation.

As described above, the configuration determines whether a single cacheis provided for an entire DVAES-enabled device, whether a plurality ofcaches is provided at the VAA level, or whether a combination of the twodistribution strategies are deployed.

The cache manager component provides active management of the cacheassociated with each VAA. Each cache manager component is started whenits respective VAA is started on a DVAES-enabled device. The cachemanager could use rule-based configuration information to determine thesize of the cache, cache update frequency, and other parameters relatedto the management of the cache and the cached materials. In someembodiments, the cache manager may be shared between VAA instances on aspecific device.

The cache manager is preferably configured to proactively review thecontents of the cache and to refresh the cached materials on the basisof predicted use. For example, a cached item that is used regularly willbe updated in the cache more frequently than an item that is not usedregularly. This approach reduces the network latency apparent to a userwhen a voice browser is performing a VA component using cachedcomponents by limiting the number of times that the cache must berefreshed while VA components are performing in real-time.

The cache manager may be configured to register a messaging interface toreceive update requests from other components of a DVAES or DVAMS. Uponreceipt of a message indicating a change in cached materials, the cachemanager automatically initiates a refresh of its cache of these changedmaterials. In most cases, the refresh operation can occur in thebackground without the user noticing the operation.

In some embodiments, the cache manager may publish a public interfaceoutside the device for use by the DVAMS, a VASS, or other devices toprovide directions to update specific materials stored in the cache.This interface can be a programmatic interface, a specific protocol, ora web-enabled interface using a protocol like SOAP over HTTP to servicethese requests.

Each voice browser provides a runtime environment for performing voiceapplications components. A voice browser comprises a voice applicationinterpreter and the necessary messaging interfaces between the voicebrowser, shared VAA services, and components such as Line Manager, ASR,TTS, cache, and OA&M Services.

There may be multiple types of voice browsers, each with their owndifferent voice application interpreters. The type of a voice browserrequired is based upon its need to access services of the VAA and theDVAES enabled device, upon characteristics of the voice applicationinterpreter, and upon instructions of the VA component being performed.As an example, a first voice application interpreter (VAI) may notsupport any telephony features, hence the only voice application thatthe interpreter could perform is one that has no telephony instructions.Alternatively, a second, more complex voice application interpreterwould support the complete VoiceXML 2.0 standard, which includestelephony standards.

Embodiments of the voice application interpreter support establishedvoice application specification standards such as VoiceXML, X+V andSALT. Additional embodiments could also support vendor-specificextensions and derivatives of such standards. Alternatively a voiceapplication interpreter may support proprietary or non-standard voiceapplication components. A voice application interpreter may additionallysupport scripting languages such ECMA Script or a similar mechanism thatextends the voice application specification with scripted logic.

The voice application interpreter provides service interfaces to VAAservices. These service interfaces are messaging conduits between aVoice Browser and the VAA services, such as the ASR service, the TTSservice, and the OA&M service. The service interfaces may be based onopen standards messaging and control protocols (e.g. MRCP for ASR), astandard services interface language such as SOAP, or the serviceinterface may be based on a specific direct API implementation.

Furthermore, a voice browser may access the Local Services Engine (LSE)to provide VA components additional capabilities, or to improve theefficiency of VA component performance. The Voice Browser will providethe ability to process VA component requests and propagate such requeststo the LSE. The interface between the LSE and the voice browser could bean API or a proprietary inter-process protocol.

A voice browser session is initiated when a new request is made by theline manager 6110 for a voice browser to perform a particular VAcomponent. Typically, each voice session is associated with one voicebrowser session, however one voice session may be associated with aplurality of voice browser sessions. In example embodiments, “Hot Word”and transcription services may be implemented by having a voice sessionassociated with both a first voice browser session, and a second voicebrowser session performing a voice application that provides the “Hotword” or transcription service.

Once a voice browser session is initiated, the voice applicationinterpreter required to process the voice application is activated. Thevoice application interpreter would then typically load and validate thefirst VA component into memory and begin to perform the VA component byinterpreting, running, or playing each of said VA's components. Aninstruction to load a VA component may be based on the configurationmaterials, user input, or an aspect of a running VA component. Someexamples of such aspects include VA component logic, a specificationwithin a VA component, a DTMF interaction with the user, or the startingof a session with a voice browser.

When a voice browser is first instructed to begin running a set of VAcomponents, the voice browser or its voice application interpreter maypre-obtain a required set of VA components to ensure that the VAcomponents are immediately available from the cache when they areneeded. In some embodiments, a voice browser or voice applicationinterpreter may pre-fetch a complete or partial set of VA components ata time prior to performing the first component of a VA, thus ensuringthat a consistent set of VA components are present in the cache of aDVAES-enabled device. The list of required VA components may be found inthe manifest. This permits the VA's performance to progress withoutdelays that might occur if VA components were obtained from a networkresource. Pre-fetching VA components eliminates a user's perception ofpoor performance by providing near-real-time VA component loads duringVA performances. Performance improvements are attained by limiting theamount of time that a user must wait for a VA component to be loaded(including collecting VA components from external servers), and insupporting immediate context switches between performing VAs.

The association between a voice browser and a VA and its components maytake the form of specifying a URI or URL. The association may be madebased upon the capabilities of the voice browser and the requirements ofthe voice application, the needs of the user, and performanceconsiderations. Furthermore, a voice browser and voice applicationinterpreter further enable the performance of a VA component byaccepting input from a voice session, processing said input inaccordance with instructions provided by a VA component, and bycommunicating instructions between a voice application interpreter and aVoice Browser based on aspects of the currently loaded VA components.Additionally, a voice application interpreter may pass service requeststo other VAA components and services

Alternatively, a voice application interpreter could also support itsown network component, such as a HTTP component, to fetch VA componentsfrom cache or remote locations. Preferably, a voice applicationinterpreter would have the ability to instruct the voice browser to playan audio source, such as an audio file which is a VA component, andreceive processed user speech, DTMF input, or telephony signalinginformation. Optionally, an implementation of the voice applicationinterpreter may support Multi Modal capabilities that can perform VAcomponents that instruct the Voice Browser to process voice input andoutput text to a screen, or process textual input and output audio to aspeaker.

VA components provide instructions to the voice application interpreter,and the voice application interpreter may in turn instruct other VAAcomponents and external services to do certain things based on theinstructions it receives from the VA component. The voice applicationinterpreter may function in an instruction only mode or in a controlmode. The voice application interpreter in an “instruction only” modepropagates performance instructions to the voice browser, and the voicebrowser then further propagates such requests to VAA services. The voiceapplication interpreter in a “control mode” functions as a voiceapplication interpreter in the instruction mode, and additionallymanages at least some VAA resources, and acts as a conduit for passingresources between VAA components.

As an example, in the instruction only mode, a voice applicationinterpreter might fetch a VA component containing an audio prompt, andinstruct the voice browser to play the audio prompt. The voice browserwould simply propagate the play instruction with the location of theloaded VA component to the line manager, who in turn would instruct theAudio and Signaling Session, which will in turn instructs a modulesupporting the hardware or network service on the DVAES enabled deviceto execute a “Play audio” request.

If a voice application interpreter was acting in the control mode, thevoice application interpreter is responsible for playing of a prompt bymanaging the buffering of the Audio to the Voice Session, henceintimately interfacing with the Audio and Signaling session. In anotherexample of the instruction mode, a voice application interpreter couldpass an instruction via the Voice Session to Audio and Signaling sessionto terminate user spoken input stream directly to the ASR Service.

The Voice Browser Voice Application Interpreter loads and performs theVA components. This performance includes the performance of theinteraction VA components and the performance of referenced resource VAcomponents (e.g. audio files and grammars) in a specified order andorganization as specified by the interaction VA components. During theperformance of the VA components, the Voice Application Interpreterperforms the component to enable interactions with the user.Additionally the VA components also have the ability to instruct theVoice Browsers to load and transition to other VA components. Ameaningful interaction is typically established by loading andtransitioning to and from many VA components. The possible permutationsand combinations of the VA component performance sequencing aregenerated by the VA in the VASS during the rendering process. Thespecific combination of VA components that are performed is typicallydetermined by the User during a Voice Browser Session. For example, theVASS may provide VA components pertaining to the “Main Menu Selection”,“Voice Mail”, and “Address book.” While these are the possible VAcomponents that a User could interact with, the specific combination ofVA components is determined during the interaction with the user, as hemay simply just navigate from main menu VA components to Voice Mail VAcomponents during a given Voice Browser Session.

As mentioned, a Voice Browser could perform multiple transitions betweenVA components. These transitions could be enabled by fetching VAcomponents from the cache that may be distributed by the DVAMSbeforehand, or may be fetched from the VASS storage in real time or maybe fetched from the VASS as a result of a rendering process by the VASSin real time. A Voice Browser session could support all such transitionsand fetches of VA components in any order. A voice browser may includecertain features that may be available for all VA components to access.These features are geared to streamline and standardize the VAdevelopment process by natively providing certain capabilities in thevoice browser that all VA components could access. The voice browser mayalso support global navigation controls that are accessible to all VAcomponents. Navigation controls include the capability for a user toissue commands like Back, Forward, Repeat, Home, Main Menu etc., and forthe Voice Browser to process such input without any performanceinstruction from the VA component. The voice navigation facility wouldbe available to all applications. In this embodiment the Voice Browserwill pass instructions to the ASR service independent of the VAcomponent performance.

Optionally the voice browser may provide comprehensive error andexception handling support to all VA components. By doing so, the VAcomponents errors and exceptions are always handled in a standard mannerwithout any special handling. A VA component may choose to override orsuppress such capabilities either via instructions in the VA componentsor via configuration parameters.

Each voice browser may provide detailed logs of internal voice browserevents to the OA&M Service. The voice browser log items may includedetails of one or more voice browser sessions, including start time, endtime, voice browser ID, line manager channel ID and type, Audio andSignaling session ID and type, VA Component instruction, Audio Played,ASR Service ID, ASR or DTMF Request, and ASR or DTMF response.Additionally, the voice browser logs may include detailed errormessages. The error logging level provided by the Voice Browser isconfigurable and may range from severe to warning.

Local Services Engine (LSE) 6130 is a VAA execution environment forpluggable utility components that provides a Voice Browser and VAcomponents additional application processing and logic support for theperformance of VAs.

In some circumstances, the Voice Browser is either unable to perform aVA component request, the VA component performance by the Voice Browsermay be inefficient, or the VAA is configured to provide performanceacceleration services for specific common services. In suchcircumstances, the LSE may provide a VA component support by offloadingcertain computing requirements, for example: date arithmetic, zip codevalidation, pass code verification, City and State verification, andother similar functions. By providing such services, the Local ServicesEngine exponentially improves the VA component performance.

In other cases, the logic of determining and controlling the order ofthe VA component performance may be provided by the LSE. For example,the decision to switch to DTMF VA components if background noise ishindering speech recognition may be initiated by the LSE. In still othercases, the Local Services Engine provides VASS-like features ofrendering VAs for key voice applications in order to eliminate anynetwork latency or dependency for performing the Voice Applications.

The LSE provides a standard pluggable interface API that will allow LSEutility components to operate. The Voice Browser may also provide LSEmessaging constructs to allow a VA to access the LSE utility components.

The LSE may propagate requests and data received from a Voice Browser tothe appropriate LSE utility components. The LSE utility componentoperates on that request. Such operations may be synchronous orasynchronous. An example of a synchronous request is an authenticationrequest to an external system. Once the requests are processed by theLSE components, the appropriate output is returned to the VA beingperformed by the Voice Browser.

The ASR Service 6140 provides Speech Recognition features to a VAA. TheASR service abstracts the interface to commercial Speech Recognitionengines and provides the VAA a standard interface to access the SpeechRecognition engines. The Speech Recognition engines are typicallylocally deployed on a DVAES Enabled device and provide a standard API orprotocol such as MRCP to provide Speech recognition features to the VAA.Alternatively, the ASR service may provide an interface to a networkbased ASR engine via standard or custom protocol. Commercial SpeechRecognition Engines include IBM, Nuance, and Sensory.

The voice browser initiates a connection with the ASR Service when VAcomponents issue a speech recognition request. Such a connection betweenthe ASR service and the Voice Browser is called an ASR Session. An ASRSession has a unique ID. The ASR could provide an instruction andtransport session, an instruction only session, or a transport onlysession.

An instruction and transport ASR session provides a voice browser theability to instruct the ASR engine to load a grammar (a VA component) byproviding a reference to the grammar, and to begin recognition uponreceiving such instruction. The ASR Service would then be waiting forthe transport of the audio stream from the voice browser. Once the audiois received, and the ASR engine processes the audio, the ASR serviceprovides the recognition results back to the Voice Browser.

Alternatively an ASR service could establish an instruction only sessionwith the voice browser. In this case, the voice browser would instructthe ASR engine to load a grammar (a VA Component) by providing areference to the grammar and the Voice Session ID with the Line Manager.The ASR Service would establish a transport only session with the LineManager to receive the audio data directly from the Line Manager. Inthis case, the Voice Browser would be functioning in instruction onlymode, and would have no control of the audio stream. The ASR sessioncould be active and persistent for the duration of the voice browsersession, hence maintaining multiple recognition contexts for the entireduration of the voice browser session. Alternatively, the ASR Sessioncould be transient and could be established and destroyed several timesduring the course of a voice browser session. Also, in some embodimentsan ASR session could be active and persistent for the duration of thevoice session. In this case, the ASR session could potentially bemaintaining multiple recognition contexts to support more than one voicebrowser session if such sessions are associated with a voice session.

The ASR service preferably supports a plurality of recognition engines.The ASR service may support multiple simultaneous ASR sessions providingSpeech Recognition services to one or more Voice Browsers. The ASRservice could also be shared between multiple VAAs or otherDVAES-enabled devices.

In some circumstances the ASR Service could provide intelligentrecognition management capabilities. These include the ability to dosimultaneous recognition of the same utterance across differentrecognition engines, which could be local or remote. The ASR servicecould also manage the ability to use an external ASR engine forspecialized (complex grammars) or higher quality speech recognition. Anexample of the above includes the capability to use a remote recognitionengine when the local recognition engine does not provide the desiredrecognition accuracy for given utterance.

In an alternate embodiment, the ASR service could provide an interfaceto a remote transcription service. The transcription service could beused in a just-in-time mode, or in an asynchronous mode. In thejust-in-time mode, the spoken utterance could simultaneously be sent tothe recognition engine and to a remote live agent/person transcriptionservice. The live agent/person transcription service may transcribe orinterpret the result and provide results to the ASR service. The liveagent/person could also be provided with a menu of potential choiceswith the audio he must transcribe. This menu could be provided by the VAin the context of what the VA expects the user's answer might be. Thisallows the operator to make a quick selection, to reduce any lag time inthe voice application. Also, the user might be played a stalling promptwhile the transcription is being accomplished. Optionally therecognition results may also be sent to the live agent/person. The liveagent/person transcription service would transcribe the utterance byhearing the utterance and provide the transcription results to the ASRService. Optionally the live agent/person transcription provider mayalso compare the transcription results with the recognition resultsprovided by the ASR service and inform the ASR Service of the results ofsuch comparison.

In the just-in-time mode, the Voice Browser would be waiting for the ASRservice to complete the transcription function before it proceeds tofurther perform the VA component. In the asynchronous mode, the ASRservice receives the utterance for transcription and informs the VoiceBrowser of such receipt, based on which the Voice Browser proceeds withthe performance of the VA component.

The ASR Service could also provide VA components with speakerverification capabilities. A voice browser could initiate a speakerverification request to the ASR service. Once the ASR service receivessuch instruction, and the corresponding user utterance, the serviceinitiates the speaker verification component of the ASR engine. TheSpeaker verification component would verify the utterance against aregistered voice print and provide the verification results to the voicebrowser. The ASR service may access an external voice print database tocomplete the verification. Alternatively the ASR service could alsoperform the speaker verification process by accessing a remote speakerverification service.

Each voice browser, through its voice application interpreter, maysupport a plurality of grammars. This could include DTMF grammars, andASR grammars. The voice browsers would preferably support somecombination of GrXML, ABNF and NLSML (Natural Language Semantic MarkupLanguage) to establish a recognition context for the recognizer toprocess spoken utterances. The recognition engine would either beprovided the grammar by the voice browser, or the recognition enginewould be provided a URI/URL reference to the grammar, in which case therecognition engine would fetch the grammar from a network source. Oncethe grammar is available, the recognition engine would compile thegrammar (if it is not pre-compiled) load the grammar, and establish arecognition context for the spoken utterance. The recognition enginemight also record the utterances and provide the recordings for usewithin the voice application.

The ASR Service 6140 would preferably provide detailed logs of internalASR Service events to the OA&M Service 6160. The ASR Service Log itemscould include details of all ASR sessions, including Start Time EndTime, Voice ASR Session ID, Browser ID, Line Manager Channel ID andType, Audio and Signaling session ID and Type, ASR or DTMF Grammar, ASRor DTMF Recognized output, confidence score, n-best list, and RecordedAudio. Additionally the ASR Service logs could include detailed errormessages. The logging level provided by the ASR Service may beconfigurable and may provide varying levels of logging in response to aconfiguration setting.

The ASR Service could also perform error notifications to the VAA's OA&MService. The error notifications could range from severe to warning, andthe detail level could be configurable.

The TTS Service 6150 dynamically converts text-based content toclose-to-natural sounding speech and provides such capability to theVoice Browser. The TTS service abstracts the interface to commercial TTSengines and provides the VAA a standard interface to access the TTSengines. The TTS engine typically is locally deployed on the DVAESenabled device and provides a standard API or Protocol to provide TTSfeatures to the VAA via the TTS Service. Alternatively, the TTS servicecould provide an interface to a network based TTS engines via standardor custom protocol. Commercial TTS Engines include, IBM, AT&T etc.

The TTS Service would receive an instruction from a voice browser toconvert Text to Audio. The voice browser would initiate a connectionwith the TTS Service when a VA component issues a TTS request. Such aconnection between the TTS service and the voice browser is considered aTTS Session. A TTS Session has a unique ID. A TTS session could be aninstruction and transport session, an instruction only session, or atransport only session.

An instruction and transport TTS Session with a voice browser providesthe voice browser the ability to instruct the TTS engine to convert textto synthesized audio, and to begin the conversion. Upon receiving suchan instruction the TTS Service would convert the text to synthesizedaudio and transport the audio back to the voice browser.

Alternatively, the TTS service could establish an instruction onlysession with the voice browser. In this case the voice browser wouldinstruct the TTS engine to convert text and transport the synthesizedaudio to a target voice session.

The TTS Service could also establish a transport only session with theLine Manager to send the audio directly to a TTS Session with the LineManager. In this case, the Voice Browser would be functioning ininstruction only mode and would have no control of the Audio Stream.

A TTS session could be active and persistent for the duration of a VoiceBrowser Session. Alternatively, a TTS Session could be transient andcould be established and destroyed several times during the course of aVoice Browser Session. Also, in some embodiments, a TTS session could beactive and persistent for the duration of a Voice Session. In such acase, the TTS session could potentially be supporting more than oneVoice Browser session if such sessions are associated with a singleVoice session.

The TTS service could support a plurality of TTS engines. The TTSService could therefore support multiple simultaneous TTS Sessionsproviding TTS services to one or more Voice Browsers.

A TTS engine could either be provided with text by a voice browser, orthe voice browser could provide a URI/URL reference, in which case theTTS engine would access the text from a network source or cache. Oncethe text is available, the TTS engine would convert it to synthesizedaudio. A TTS engine may also store the synthesized audio in a cache, orsomewhere on a network resource.

The TTS Service would provide detailed logs of internal TTS ServiceEvents to the OA&M Service. The TTS Service Log items could includedetails of all TTS sessions, including Start Time, End Time, VoiceSession ID, Browser ID, Line Manager Channel ID and Type Audio,Signaling session ID and Type, text, and a resulting synthesized audiofile. Additionally the TTS Service logs could include detailed errormessages. The Logging Level provided by the TTS Service may beconfigurable and may provide varying levels of logging in response to aconfiguration setting.

The TTS Service could also perform error notifications to the VAA OA&MService. The error notifications could range from severe to warning, andthe detail level may be configurable.

The Operations, Administration, and Monitoring service of the VAA isresponsible for ensuring that the VAA components are workingefficiently. The OA&M Service is also the primary VAA component thatinterfaces with the DVAMS.

Upon start up, the OA&M Service loads the configuration materials andestablishes a connection with the DVAMS. The OA&M service could operatein an active mode and/or a passive mode. In the active mode, the OA&Mservice starts all the other services in the VAA based on the orderspecified in the configuration data. In passive mode, all the VAAServices self-start based on a startup routine in the OS. Once theservices have started, they register with the OA&M.

The interface between the OA&M Service and the various other VAAservices may be based on an API or a messaging protocol. Examples ofmessaging protocols that may be used include SNMP, RPC, SOAP, and TCP/IPMessaging. The connection between the OA&M service and the DVAMS mayalso be based on a network provisioning, communications, and monitoringprotocols or specifications like SNMP or tr-069.

The OA& M Service, based on an external instruction, such as one from aDVAMS, or a DVAES Device or service condition, may shutdown and re-startthe VAA components and services. Device and service conditions includesuch items as CPU load, available memory, and changes in configuration.The OA&M service may notify services to reload changed configurations asan alternative to a service shutdown and restart.

The OA& M Service may receive and store log and error events receivedfrom the VAA components and services. The OA& M service may propagatesuch log and error information to the DVAMS and optionally to anadditional Network management system. Additionally the OA&M service maysend a health heartbeat signal to the DVAMS.

The OA&M service may continue to function if the DVAES-device istemporarily disconnected from the network. The OA&M Service would cachenormal real-time logs until a connection is available. If the cached logsize is too large, extra logs are purged as necessary to free up spaceto record the logs.

VAA's may be clustered to provide redundancy, to distribute processingloads, and to optimize the use of specific resources. In someembodiments, VAA services may be provided by a plurality ofDVAES-enabled devices, with the dispatch of specific voice sessions toany of a plurality of VAA instances operating on disparate DVAES-enableddevices. By utilizing the voice session transport mechanisms in thismanner, VAA services may be provided by whichever DVAES-enabled deviceis able to best provide the requested services at a specific point intime.

In addition, VAA services may be provided on specific hardware platformsin which specialized hardware assistance is provided. For example, if aDVAES includes a high-end server that provides complex speechrecognition, a VAA may be configured to attempt local speech recognitionon the local hardware, but to send complex or unrecognizable speech tothe high-end server for further analysis. Upon receipt of the resultsfrom the high-end speech recognition server, the VAA may continueprocessing of VA components locally.

A VAA registers itself with a DVAMS when it is first instantiated. Theregistration process associates one or more specific DVAES-enableddevices, and the capabilities of those DVAES-enabled devices, with aspecific VAA. The registration also signifies that the VAA instance isavailable for use within one or more VDAEs, which will be explained ingreater detail below.

In some embodiments, a VAA may be present on a DVAES-enabled device whenthe device registers. However, the VAA itself needs to register with theDVAMS to receive the appropriate membership credentials and to make itavailable so VAs and users may be allocated to it. In other embodiments,“new” VAAs are instantiated, either by pushing them from a centralrepository, or by instantiating a “new” VAA (or portions thereof) on aDVAES-enabled device by downloading VAA materials. Each “new” VAAregisters with a DVAMS upon instantiation, and provides information tothe DVAMS regarding its capabilities, resources, and its associationwith specific DVAES-enabled device(s).

FIG. 10 illustrates an exemplary set of steps taken by a VAA in order toregister itself. This process assumes the VAA is being instantiated on aDVAES-enabled device that has already registered with the DVAMS.

In step 8110, the selected VAA starts. Next, in step 8120, the selectedVAA does a self-check to determine if it is registered. The self-checklooks for materials provided by a DVAMS when the VAA is registered, andif found, uses these materials to determine if the VAA is registered. Ifthe VAA is already registered, the process is complete. The VAA mayoptionally perform an integrity self-check to ensure that it is completeand has not been tampered with. If the integrity self check fails, theVAA shuts down without further processing.

Assuming the VAA has not already been registered, in step 8130, the VAAcontacts a DVAMS. The DVAMS used may be the same DVAMS with which thedevice itself registered, or a different DVAMS. The DVAMS to use isspecified in the configuration materials or certification materials forthe device and VAA.

Next, in step 8140, the VAA provides the DVAMS with its device ID, VAAID, and device/VAA configuration information. If the VAA does notalready have a VAA ID, one is generated by the VAA using the uniquedevice ID. In step 8150, the DVAMS returns membership materials to theVAA, which bind device ID, and VAA ID to the VAA. These materials may beprovided by the DVAMS directly to the device, or it may be provided to adistribution mechanism from which they are subsequently downloaded bythe device.

In step 8160 the VAA stores VAA registration materials in VAAconfiguration materials. Finally, in step 8170, the VAA downloads anyrequired VA components specified by the VAA registration materials.Further, if a local service component is required by the VAAconfiguration materials, the VAA startup process starts the localservice component

If a DVAES-enabled device has multiple VAAs configured on it, theregistration process repeats for each VAA on the device. Note, thisprocess can repeat for each user and VAA on the device. Different usersmay be associated with different VAAs, and with different DVAES, or asingle user may be associated with a plurality of VAAs in the device.Associations between the device, user, and VAA are managed between thedevice configuration materials and the respective DVAMS's.

Each VAA also starts its own line manager, which in turn starts thedefined voice browsers and voice applications defined in theconfiguration materials. If no startup voice browsers are defined in theconfiguration materials, a VAA may not initiate a voice browser uponbooting and functions as a telephony gateway.

If a VAA-specific caching component is required by the VAA configurationmaterials, the VAA startup process also starts the cache manager andcache interface, as necessary. Changed items in the cache are refreshedduring the booting process as required by the appropriate cache rules.For example, if a user changes his voice application preferences on aremote server, the changes are propagated to the DVAES-enabled device aspart of the requisite re-rendering and propagation of the user'spersonalized VAs.

We will now turn to a typical process which would occur when a userwants to run a voice application. References will be made to FIG. 11 ,which illustrates this method.

In step 7110, a user action occurs to initiate communication with theDVAES-enabled device. This could occur when the user lifts a telephonehandset off hook on a connected telephone device, or possibly by pushinga physical or virtual switch on a telephone device or a microphone tosignal that the user wants to initiate activity. This could also occurif a call is received from a phone network, whether that be a PSTN orvia a VoIP network. Further this could occur if a user speaks a key wordor phrase that signals that the user wishes to initiate activity.

In step 7120, the DVAES-enabled device determines the type of audio andsignaling session. In steps 7130 and 7140, the DVAES-enabled deviceassociates an audio and signaling session with a specific VAA on thebasis of the configuration materials for the device and/or VAA. Thisassociation is a voice session. The DVAES-enabled device performs alookup of configuration materials as part of the process to associatethe session with a specific line manager, based upon attributes of thesession such as the hardware device, ANI, or other attribute.

In steps 7150 and 7160, the line manager establishes a connectionbetween the voice session and a voice browser, which creates a voicebrowser session. The line manager determines which voice browser and VAto run (although in alternate embodiments, the user may make thisdetermination) based on configuration data or dynamic rules (e.g., rulesbased on hardware interface, phone line, date, time, etc.). In someinstances, the voice browser is already running and the VAA connects thevoice session to an already running instance of the voice browser. Forexample, a voice browser may be preconfigured to listen for “hot words”and DMTF instructions. This voice browser may be left running betweenuses to reduce the amount of time spent stopping and restarting a voicebrowser with the same voice application. In other instances, the VAA maycreate a new instantiation of a voice browser to handle the VA.

In step 7170, the voice browser fetches materials as necessary for thevoice application. The running voice browser fetches (if needed) thespecified voice application components from the cache mechanisms of theVAA, or possibly from a VASS or CDS, in accordance with the DVAMSprovided content management rules. Voice application components,including required resources, audio files, and data may be located inVAA cache, device cache, local storage, a CDS, a VASS, a DVAMS, and/or athird party network.

Finally, in step 7180, the voice browser performs the VA to enableinteractions with the user.

The Content Distribution Service

The Content Distribution Service (CDS) may be deployed in strategiclocations of the network. The CDS is an optional component of a DVAESthat may be helpful to overall system performance when the DVAES userbase substantially increases. The CDS provides network-based caching ofcontent such as VA components, audio files, grammars, etc. in thebroadband service provider's network. This caching helps performance byreducing network traffic latency by moving static content closer to thenetwork edge.

Content Distribution Service systems are well understood by thoseskilled in the art, and are provided commercially by companies such asAkamai Technologies, Boston Mass.

The Distributed Voice Application Management System

A Distributed Voice Application Management System (DVAMS) is responsiblefor operating, monitoring, and administering a DVAES. The DVAMS alsoprovides device, user, and DVAES-component registration services, andprovides administration support for configuring one or more VirtualDistributed Application Environments (VDAEs) that may be deployed usingspecific DVAES implementations. The concept of a Virtual DistributedApplication Environment is discussed in greater detail below.

The DVAMS hardware includes proxy servers, web servers, applicationservers, load balancers, and other commercially available systems thatare integrated to accomplish the goals, functions and purposes of theDVAMS. Foundation software for a DVAMS includes standard server basedcomputing technologies such as Dot-Net, PHP, and Java.

Some of the functions performed by the DVAMS include:

-   -   1. Real time voice application management, which includes voice        application installation, activation, deactivation, monitoring,        and parameter configuration.    -   2. System monitoring, which includes monitoring hardware and        third party software, monitoring for errors, warnings and        notifications.    -   3. System configuration, which includes setting parameters and        configuration files, executing recovery routines, and platform        image rollback capabilities.    -   4. Allocation of resources to individual users, DVAES-enabled        devices, and services to effect a smoothly operating DVAES. This        allocation of resources could help to establish one or more        Virtual Distributed Application Environments (VDAEs), which are        discussed in more detail below.    -   5. Collection and analytical processing of system data produced        during the operation of each DVAES. This data primarily includes        system configuration settings and information collected during        runtime from various DVAES services and components. For        instance, the data could include monitoring results, tuning        logs, and error notifications. As noted above, this data can be        analyzed and used during the VA rendering process to help        customize or personalize individual rendered VA components that        are allocated to particular users.

One of the important functions performed by the DVAMS is theregistration of users, DVAES-enabled devices, VAAs and VAs. The DVAMSmay maintain one or more registration directories to track registrationinformation, and associated credential information. The DVAMSregistration directories can function to integrate and publishinformation about a user, a group of users, devices, VAAs, VAs andVASSs. The registration directory may be constructed using a commercialdirectory product such as openLDAP (open source). Alternatively,commercial directory services such as those provided by Novell (ProvoUT) may be used. In other embodiments, a database such as thosecommercially available from Oracle or Microsoft may be used.

As explained in more detail below, the DVAMS Registration Service couldbe a web-based interface, or the Registration service could functionthrough one or more VAs. One preferred implementation of the DVAESregistration service is a SOAP-based web service, although other formsof service may be provided as implementation requirements dictate. Theregistration service accepts a request for registration from aDVAES-enabled device, a user, a group of users, VAAs, and VASSs,validates this request, and if the request is valid and authorized,enters the registration information in the DVAMS registration directory.

FIG. 12 shows the components of an exemplary embodiment of a DistributedVoice Application Management System (DVAMS) 11000. The DVAMS comprisesPresentation Services 11110, DVAMS Services 11200, and MessagingServices 11300.

The Presentation Services 11100 include the components of a DVAMS thatprovide user interfaces, service and administration interfaces foroperations personnel, and other public interfaces to DVAMS and DVAESservices. Specifically, the presentation services can include a ServicesAdministration Portal 11110, a Network Operations Portal 11120, and aSubscriber Portal 11130.

The Service Administration Portal 11110 would be used by Customer Careand DVAES Operators to manage a DVAES, its voice applications, andusers. The Service Administration Portal would typically be a web basedsystem that provides user and device management features. The ServiceAdministration Portal will also facilitate provisioning of applicationfeatures by establishing configuration parameters and applicationpreferences for different users.

Operators will use the Service Administration Portal to create users andprovision the different application features available to the user. TheService Administration Portal could also provide an operator the abilityto organize users into groups and to assign roles and permissions to theusers and groups for the purpose of administering the applicationsavailable to the users/groups of users. The users could be grouped basedon regional locations, or communities, or based on their membership in aparticular organization.

An operator may enable a device associated with a user using the ServiceAdministration Portal. Activation of the device will allow the user toaccess the different voice applications using the device.

The Operator may additionally create one or more VDAEs using the ServiceAdministration Portal, and assign users, applications and associateddevices to particular VDAEs.

The Service Administration Portal could also be used by an operator togenerate reports and device monitoring and error notificationinformation that would facilitate management of the applications anddevices associated to a user.

The Network Operations Portal 11120 could be used by Network OperationsCenter (NOC) personnel to manage and monitor the underlying hardware andsoftware elements of the DVAES. Network-centric monitoring applicationsare well understood by those skilled in the art. The Network OperationsPortal could also be a web-based application.

The Subscriber Portal 11130 provides a personalized environment forusers to manage and access their voice applications and deviceconfigurations. The subscriber Portal might also allow new users toregister for services, as described below. Additionally, the portal mayact as a medium for operators to provide users with enhancedapplications in the form of pluggable user interface components called“portlets.” The Subscriber Portal may also facilitate the provision ofpromotional information and customer support to the users.

In some embodiments, users may be able to manage aspects of theirDVAES-enabled devices, such as the device configuration settings. Inaddition, the Subscriber Portal may allow a user to add or subscribe tonew VAs, to modify their existing VAs, and to cancel VAs.

The Subscriber Portal may also provide users with the ability tocustomize various VAs based on user preferences, and for the user toassociate themselves with one or more DVAES-enabled devices. Thecustomized VAs would then be used by one or more DVAES-enabled devicesto provide a personalized user experience.

One example of how a user might customize a VA would involve a VAdesigned to provide the user with messaging via e-mail, or voice mail.The user could customize the messaging application to greet callers withpersonalized messages, or provide call handling of incoming calls basedon different caller profiles. In other embodiments, the user may be ableto customize e-mail messaging applications to notify the user of thereceipt of important e-mails by ringing a phone connected to the deviceand playing an alert message. Of course, the Subscriber Portal couldallow a user to customize VAs in many, many other ways.

As noted above, the Subscriber Portal 11130 may allow a user to registerwith a DVAES. This would typically be a web-based portal that a usercould access to register for services. FIG. 13 illustrates a process inwhich a user registers and activates their account using a web-basedSubscriber Portal of a DVAMS.

In step 17110 the User clicks on the new customer registration link. Instep 17120, the link takes the user to the new customer registrationpage. This page is hosted/controlled by a DVAMS.

In step 17130 the user inputs personal information via the webinterface. The Personal information could include Name, Address, PhoneNumber, a Password, a list of Device ID(s) (as provided by the devicedocumentation), service selections, and additional personalizationinformation.

In step 17140 the DVAMS would create a user ID, if necessary. Asmentioned above, a user could be authenticated in various differentways, including through a voice print analysis. Thus, the input orgeneration of a user ID and password may not be necessary.

In step 17150 the DVAMS checks to see if a DVAES-enabled local devicewhich is available to the user is already registered. If not, theprocess would terminate. Of course, the user could always return to theportal after a device available to the user has been registered. Also,as explained below in connection with FIG. 14 , once the user isregistered with the system, the user could access the DVAMS using anactivated and registered DVAES-enabled device to associate himself withthe device.

If the result of the check in step 17150 indicates that a DVAES-enableddevice available to the user has already been registered, then in step17160 the DVAMS produces configuration materials, including bindingmaterials for user/devices. In optional step 17170, the DVAMS may alsoproduce other, optional configuration materials. In step 17180 the DVAMSpushes he configuration materials to a directory. A notification mayalso be sent to an associated VASS of this change in the systemconfiguration, which would cause the VASS to render personalized VAs tothe device associated with the user. In step 17190, the DVAMS would pushconfiguration materials to the device itself, possibly via a CDS. Anoptional notification may be sent to the device or DVAES components tofacilitate the pushing of the materials from the CDS to the device.

As noted above, if a user has been registered and activated, and theuser needs to associate himself with a DVAES-enabled device, theassociation could be accomplished using a VA. This might be necessarythe first time that a user subscribes to the DVAES, or the user mightwish to associate himself with a second or additional device in theDVAES. For this to be possible, the user would need access to aDVAES-enabled device, and that DVAES-enabled device and a VAA on thedevice would need to be registered and connected to the DVAES. FIG. 14illustrates such a process.

In step 16110 the VAA would connect to a registration service in aDVAMS. Next, in step 16120, the DVAMS would identify the VAA fromconfiguration materials. In step 16130 the DVAMS would bind the VAA tothe user. Then, in step 16140, the DVAMS adds or updates the userinformation in a registration database.

Note, the processes of registration of a DVAES-enabled device, a VAA,and optionally a user may all be combined in a single process.

Returning now to a description of the other elements of the DVAMS, theMessaging Services 11300 of a DVAMS could comprise, among other things,a Device Messaging component 11310, a VAA Messaging component 11320, aVASS Messaging component 11330, and an External Messaging component11340.

The Device Messaging component could operate to send notificationmessages to DVAES-enabled devices. This could include messages regardingcache updates and refresh notifications. In addition, the DeviceMessaging component could function to receive messages fromDVAES-enabled devices. Received messages could include device operationlogs, and error messages.

Likewise, the VAA Messaging component 11320 could send messages to VAAsregarding cache update and refresh notifications, or regarding the needto receive new VAs. The VAA Messaging component could also function toreceive and process messages from operating VAAs, including operationslogs, and error messages.

The VASS Messaging component could operate to send messages to VASSsregarding such things as the need to render new VA components forparticular DVAES-enabled equipment, or the need to re-render VAcomponents to a particular user's DVAES-enabled equipment to furthermodify or personalize the VA. The messages could also direct VASSs totransfer or copy various items of content into one or more CDSs. TheVASS messaging component could also operate to receive messages from theVASSs, such as operations logs and error messages.

The External Messaging component 11340 could be used to send messages tovarious users, and/or to various third parties, such as serviceproviders that are available to users via the DVAES. For instance, audioand/or e-mail messages could be sent to users to advertise theavailability of VAs. The decision to generate these sorts of messagescould be based on the user's patterns of use, or their personalcharacteristics. Likewise, the External Messaging component couldgenerate and send messages to third parties, like service providers, toinform them of new DVAES capabilities, or the existence of newuser-subscribers who might be interested in using their services. Orcourse, the External Messaging component could be used to send anynumber of other sorts of messages to users and third parties.

The DVAMS Services component 11200 can be broadly divided into DeviceServices 11210, VAA Services 11220, VA Services 11230, and VASS Services11240.

The Device Services component 11210 could include, among other elements,a Device Provisioning and Upgrades component 11212, a Monitoringcomponent 11214, and a Configuration component 11216. These servicesprovide support for device registration, deployment, and device tests,as well as monitoring and configuration services. The DeviceProvisioning and Upgrades component 11212 could include deployment andboot testing for a DVAES-enabled device, its OS, firmware, networking,and audio and signaling layers. The Monitoring component could functionto monitor device heartbeat signals, device logs, device errors, deviceCPUs, and device memory and cache allocations. The Configurationcomponent 11216 could function to manage and control device settings,device start-up, and device shut down.

The Device Services component 11210 would typically provide a serviceinterface to a device registry which enables the registration ofDVAES-enabled devices within the DVAES architecture. Registration is aprocess by which information about a specific DVAES-enabled device iscollected and stored in the device registry, and the device is issued aDVAES credential that indicates that the device has been registered.Information collected about each device could include the device'sunique ID and device configuration materials, including componentversions present on the device, hardware capabilities, and the like.Examples of the type of materials collected are included in the list ofdevice capabilities described elsewhere.

Once information about the device has been collected, it is storedwithin the device registry. The device registry can be a unique registrydedicated to the management of device information, or it may be a moregeneral service such as a DVAMS registration directory.

After the device has been registered, a DVAES credential is constructedthat indicates that a particular device has been registered with aDVAMS. The DVAMS credential may indicate the device's unique ID, that itwas registered with a particular DVAMS, and also may indicate anexpiration date after which the credential is no longer valid. Otherinformation also may be included in the credential. The DVAES credentialis returned to the device as an indication that it has been successfullyregistered.

The service interface to the device registry that is provided by theDevice Services component 11210 could take at least two forms. First,the service interface could include a web user interface that wouldallow DVAES operators to manually register and manage DVAES-enableddevices and associated information stored within a device registry. Theservice interface could also function as a fully or partially automatedinterface that allows devices to directly register with a DVAMS oncethey are connected to the network. Such an automated service interfaceto the DVAMS's device registry could take the form of RPC, SOAP, orother well-known service interface mechanisms.

A device registration process that would be conducted by the DeviceServices component 11210 is depicted in FIG. 15 . Such a process wouldtypically be performed by the DVAMS in response to a “register device”request.

First, in step 12110, the DVAMS would receive the registration request,along with the device's unique ID, and the device'scapabilities/configuration materials. Optional information included withthe request may also include an account/password or other materialsindicative of the business relationship between the DVAMS operator andthe device's owner.

Next, in step 12120, the DVAMS makes a determination if the materialsprovided by the device need validation. If so, the method proceeds tostep 12125 where the materials are checked to determine if the devicecan be registered. If the device cannot be registered, the processterminates with a failure to register the device. If the device can beregistered, or if no materials requiring validation were provided in thefirst place, the method proceeds to step 12130, where the DVAMS storesthe device's information in the DVAMS device registry. If informationabout this particular device was already present in the registry, theDVAMS replaces the contents in the registry with the newly providedinformation.

In step 12150, the DVAMS creates a device credential. In one embodiment,the device credential is a SAML assertion that binds the device's uniqueID with the DVAMS that registered it. Note, the device credential maybind the device to another DVAMS if so required by the architecture.This would result in the device being registered by a first DVAMS, butbeing controlled by a second DVAMS. A copy of the credential may beoptionally stored in the device registry.

In step 12160 the DVAMS makes the device credential available to thedevice. This could be accomplished by having the DVAMS directlydownloading the credential to the device, or by publishing thecredential to a CDS and informing the device of the location of thecredential.

In an optional additional step 12170, the DVAMS may check the device'sconfiguration to determine if the device has all requiredDVAES-enablement components. If not, method may proceed to step 12172,during which the DVAMS would instruct the device as to the additionalcomponents it requires. This instruction from the DVAMS could alsoinclude information about where the device can acquire the additionalcomponents. Then, in step 12174, the device would acquire the additionalrequired components. Steps 12172 and 12174 could also be accomplished byhaving the DVAMS simply downloading the required components into thedevice.

In other alternate embodiments, the results of steps 12172 and 12174could be accomplished by having the DVAMS create a new instance of thedevice's component list, and then publishing that list to a CDS. TheDVAMS would then notify the device that the component list has changed,and require the device to download the new component list from the CDS.The device could then download any missing components.

Returning now to the description of the elements of the Device Servicescomponent 11210, the Device Monitoring component 11214 could function tomonitor device heartbeat signals, device and operating system errorreporting, and resource utilization, including utilization of CPU andmemory. The DVAMS device-monitoring component preferably comprises aninstance of a heartbeat service, an SNMP trap sink for error reporting,and an SNMP-based resource-monitoring component.

The Device Configuration component 11216 provides configurationmanagement of devices, including management of service configurations,which comprises two aspects, configuration collection, and themanagement of the configuration section of the device registry.Configuration collection may be provided using SNMP, TR-069, or otherprotocols for collecting configuration materials from a device. Onceconfiguration materials are collected, they are associated with a deviceand stored in the device registry for subsequent use.

The VAA Services component 11220 could include, among other elements, aRegistration, Activation, and Deactivation component 11222, a Monitoringcomponent 11224, and a Configuration component 11226. The functions ofthe Registration, Activation, and Deactivation component areself-explanatory. The VAA Monitoring component could function to monitorvarious VAAs aspects, such as VAA heartbeat signals, VAA applicationlogs, VAA recognition logs, and VAA errors. The VAA Configurationcomponent could function to enable VAA extensions, VAA lines, VAAcodices, and recognizer tuning parameters.

The DVAMS VAA services component provides a service interface to a VAAregistry, which enables the registration of VAAs on DVAES-enableddevices within the DVAES architecture. The service interface to the VAAregistry could take at least two forms. First, a web user interface forDVAES operators would permits DVAES operators to manually register andmanage VAAs on specific DVAES-devices and associated information storedwithin a VAA registry. Alternatively, the service interface could be afully or partially automated interface that allows VAAs to directlyregister with a DVAMS. An automated interface to the DVAMS's VAAregistry may take the form of RPC, SOAP, or other well-known serviceinterface mechanisms.

The service interface provides the capability to “register” a VAA.Registration is a process by which information about a specific VAA andits underlying device is collected and stored in the VAA registry, andthe VAA is issued a DVAES credential that indicates that the VAA hasbeen registered. Information collected about each VAA includes the VAA'sunique ID, VAA and device configuration materials, including componentversions present on the device, hardware capabilities, and the like.Examples of the type of materials collected are included in the list ofdevice capabilities described elsewhere.

Once information about the VAA has been collected, it is stored withinthe VAA registry. The VAA registry can be a unique registry dedicated tothe management of VAA information, or it may be a more general servicesuch as a DVAMS registration directory.

After the VAA has been registered, a DVAES credential is constructedthat indicates that a particular VAA has been registered with a DVAMS.The DVAMS credential indicates the device's unique ID, the VAA uniqueID, and that the VAA was registered with a particular DVAMS. Thecredential may also indicate an expiration date, after which thecredential is no longer valid. Other information may also be included inthe credential, as desired. The DVAES credential is returned to the VAAas an indication that it has been successfully registered.

A process such as the one shown in FIG. 16 is performed by the DVAMS inresponse to a “register VAA” request. This process would typically beperformed by the Registration, Activation and Deactivation component11222 of the VAA Services component 11220.

In step 13110, the DVAMS receives the request, the device's unique ID,the VAA's unique ID, and the device's capabilities/configurationmaterials. In step 13120 the DVAMS makes a determination about where theVAA can be registered. The decision may be made based on a number offactors, including aspects of the device the VAA is running in, detailsabout the owner of the device, or other factors such as performance andthe ability of a DVAES to support the particular device or deviceconfiguration.

In step 13130 the DVAMS stores the VAA's information in the DVAMS VAAregistry. If the VAA was previously registered, the process replaces thecontents in the VAA registry with the newly provided contents.

In step 13150 the DVAMS creates a VAA credential. In one embodiment, theVAA credential is a SAML assertion that binds the device's unique IDwith the VAA's unique ID and with the DVAMS that registered it. Note,the VAA's credential may bind the device to another DVAMS. In thisinstance, a first DVAMS would have registered the VAA, and a secondDVAMS would be responsible for future monitoring and control of the VAA.A copy of the credential may be optionally stored in the VAA registry.

In step 13160 the DVAMS makes the VAA credential available to the VAA,either by downloading it to the device, or by publishing the credentialto a CDS and informing the device of the location of the credential.

In step 13170, the DVAMS reviews the VAA configuration. Then, in step13180 the DVAMS determines if the VAA has all required components. Ifall required components are present in the VAA, the process ends.However, if the result of the check in step 13180 indicates that allrequired components are not present in the VAA, the method proceeds tostep 13185, wherein the DVAMS instructs the device as to the newcomponents it requires. This instruction may be performed by downloadingrequired components directly into the VAA, or by providing to the VAA alist of the required components, or by other means.

The result of step 13185 could also be accomplished by creating a newinstance of the VAAs component list, publishing that list to a CDS, andthen notify the VAA that the component list has changed. The VAA couldthen download the new component list, download any missing components,and then restart to load the new components.

Returning now to a description of the elements of the VAA Servicescomponent 11220, the VAA Monitoring service 11224 could include aninstance of a heartbeat signal monitoring service and centralized logcollection from the VAA. Log collection may be undertaken using a commonlogging protocol such as syslog, or may be implemented using a customlog management process. ASR and TTS logs are also managed in this way.

The DVAMS receives performance logs, VA logs, TTS logs, and ASR logsfrom VAAs, and performance metrics from DVAES-enabled devices. Theselogs are processed by the DVAMS to identify errors and non-optimumperformance. Based on these analyses, changes may be made in thepersonalization materials associated with a user, a VAA, or a device(depending upon type of change).

For instance, based on the analysis of collected information, the DVAMSmay generate a new ASR grammar to correct speech recognition failuresfor a particular user. The DVAMS would then associate the newlygenerated ASR grammar with a user's preferences so as to correct ASRdeficiencies in all newly rendered VA components. The DVAMS might alsocause previously rendered VAs to be re-rendered to incorporate the newlygenerated ASR grammar.

In a second example, if VA logs indicate that a user consistently makescertain selections during the processing of a voice script, the DVAMSmay act to change the presentation order preference associated with thevoice script for that user's rendered copies of the VA.

In a third example, if the DVAMS recognizes that there are a significantnumber of cache misses when performing voice applications on aparticular DVAES-enabled device, the DVAMS may take corrective action byadjusting the cache rules to provide additional cache space and to holdVA components in cache longer.

The above examples are merely a few of the ways that the DVAMS can usecollected information to improve the processing and performance of aDVAES.

Returning now to a description of the elements of the DVAMS Services ofthe DVAMS, the VA Services component 11230 could include, among otherelements, a VA Distribution and Activation component 11232, a VA CacheContent Manager 11234, a VA Monitoring component 11236, and a VAConfiguration component 11238. The VA Distribution and Activationcomponent could function to control rendering, activation, deactivation,and registration of VAs. The VA Cache Content Manager would function toorganize content for caching. The caching could be based on varioussystem events. The VA Monitoring component could function to monitor andlog user usage for billing, VA heartbeat signals, and/or VA applicationlogs. The VA Configuration component would function to control VApermissions, VA privileges, and default behavior. Some of these itemswould be controlled or configured based on individual user'scharacteristics, their usage patterns, and their stated preferences.

The VASS Services component 11240 could include, among other elements, aSystem Data Services component 11242, a VASS Monitoring component 11244,and a VASS Configuration component 11246. The System Data Servicescomponent could provide a VASS with access to DVAMS data (e.g., logs,User billing information, Class of Service, user characteristics, etc.).The VASS Monitoring component could operate to monitor VASS heartbeatsignals, VASS errors, VASS logs, traffic, server load, CPU, and memoryusages. The VASS Configuration component could operate to controlvarious VASS parameters and data access locations.

In alternate embodiments, the DVAMS could be configured such that the VAdeployment and activation component 11232, the VAA registration,activation and deactivation component 11222, and possibly the VASSconfiguration component 11246, are all organized under a consolidatedDVAMS distribution service. The DVAMS distribution service would beresponsible for moving the VA components and the VAA components totarget locations.

A deployment specification would inform the distribution service toeither move VA and/or VAA content to the CDS, or to move the content toa DVAES enabled device, or a combination of the above. The distributionservice could process the move instruction by physically moving thecomponents to the target destinations in a push model, or by instructingthe targets to refresh themselves from a storage location. Thedistribution service may also have the ability to interface with cacheservice on the DVAES enabling layer of a VAA and/or with the VAA cacheservice.

In other embodiments, the DVAMS may have a deployment service that isresponsible for deploying VAAs to DVAES-enabled devices, and alsopossibly VAs to a VASS.

The deployment of VAAs could be triggered when a user registers with thesystem, when a DVAES-enabled device registers, or by a deploymentspecification identifying a required VAA. Upon the occurrence of atriggering event, the DVAMS deployment service would begin deployment bydistributing VAA components and VAA configuration materials toDVAES-enabled devices. The VAA components may initially be stored in aDVAMS storage area such as a registry.

The DVAMS may use the above-mentioned distribution service for deployingthe VAA components and VAA configuration materials on DVAES-enableddevices. First, the DVAMS would create and provide a deploymentspecification to the distribution service. The deployment specificationfor a VAA may include information about the VDAE, the location of a CDS,a deployment model (pull/push), a list of VAA components that need to bedeployed, and other DVAES specific deployment considerations (forinstance, the component Packing and Unpacking modes supported by theDVAES OS). The DVAMS may, in some circumstances, may only deploy the VAAconfiguration materials or a few VAA components.

The deployment of VAs could be triggered by an automated instruction, ora manual instruction from an operator which is input via the ServicePortal. The VA may initially be stored in the DVAMS storage area. Uponthe occurrence of a triggering event, the deployment service wouldgenerate VA deployment specifications and move the VA and the deploymentspecifications to the VASS storage area. The deployment specificationsfor a VA may include instructions for the VASS that are used in therendering process (for instance, a location for the rendered VAcomponents, DVAES constraints etc.). In addition, the VA deploymentspecifications could also be used by the DVAMS distribution service.

As mentioned, the DVAMS manages many broad categories of data. The DVAMScan use this data as part of an analytics process, which is continuous,which is intended to constantly improve the customization of VAs forindividual users. The objective of this analytics process is tointelligently correlate data generated by the DVAES during itsfunctioning with historical DVAMS data, and with the data stored in allrepositories, to improve the usability of the Voice Applications. Theimprovement to user experience could be specific to a user, a group ofusers, a specific device, a VDAE, or some combination of the above. Theimpact of the analytics could be to render VA components, or the VAA, orin some cases even the VASS and the DVAMS itself.

The analytics are configured to detect that a specific user is notgetting a desired usability result from a VA or a group of VAs. Thiscould be due to a variety of factors such as speech recognitionaccuracy, VA user interface design flaws, device performancedegradation, a particular input device not operating properly with aspecific user, user voice accents, certain other user demographics, anda variety of other factors.

The analytics engine correlates available data about a user, andpossibly data about that user's local devices, with other pertinentsystem data to determine root causes of any problems. In some cases, aVA might be provided for the user to collect information from the userto further narrow or determine root causes of lack of performance. Forexample, the analytics process might determine that the device under useis constantly low on memory and/or CPU, hence it does not perform wellon large grammar recognitions. The analytics might determine that thedevice is used in an environment that has very poor background noise, orthat the user is saying a phrase that is not supported by the grammaretc. Based on such results, the Analytics engine will take correctivesteps to personalize the VA or the VAA to resolve such issues.

These steps taken to resolve any detected problems could include achange in a component of the VAA, such as an acoustic model of an ASRengine, an alteration to a grammar or audio resource VA component, achange in the order of VA components that are performed, an addition ofmore error recovery and management features to VA components, or avariety of other steps.

The analytics engine could also operate to render highly personalized VAcomponents in a more proactive manner to improve user performance. Theanalytics engine could decide that the user's skill level has changedbased on observing the user's usage pattern. As a result, the analyticsengine may determine that the user should use a more intuitive andstreamlined user interface, instead of a verbose interface. The enginecould also change the size of the cache of the VAA based on how the restof the resources on a device are utilizing the memory.

Also, the analytics process may initiate content distribution to thedevice proactively to eliminate latency. For instance, the analyticsengine may determine that a user accesses certain types of content atapproximately the same times each day. For instance, the user may accesssports team scores or news headlines at approximately the same time eachday. In that instance, the analytics engine will ensure that the desiredcontent is proactively cached on the user device in advance of the timethat the user would typically access the data, which should make thedelivery of the information to the user more rapid.

Having now described the major elements of a DVAES embodying theinvention, the discussion will now turn to how the various describedelements can be organized and configured to efficiently provide servicesto users.

The first concept to introduce is a Virtual Distributed ApplicationEnvironment (VDAE). A VDAE is essentially a logical grouping of:

-   -   1. A set of users, or a plurality of groups of users; or    -   2. A set of DVAES-enabled equipment; or    -   3. A set of DVAES-enabled equipment and/or VASSs and/or CDSs; or    -   4. A grouping of any combination of the above-mentioned        elements.

The purpose of creating a VDAE is to provide a logical connectionbetween VDAES users and elements to facilitate the management of theusers and elements. The best way to illustrate the benefits of creatingVDAEs is to provide some specific examples.

In a first example, assume that multiple employees of a largecorporation are provided with various defined services of a DVAMS aspart of their job with the corporation. Assume that the DVAMS servicesare provided from Provider X. This would mean that each employee wouldhave one or more VAAs associated with Provider X, and that those VAAscould be resident on devices located in their offices, in their homes,and possibly also on some mobile computing devices. Each of these VAAswould have VAs that provide the employees with services related to theiremployment with the corporation.

Also assume that some of those employees have separately obtainedadditional DVAMS services for their personal use, and that thesepersonal use services are obtained from Provider Y. This would mean thatthe employees would also have additional VAAs associated with Provider Ythat are also stored on some devices located in the employees' offices,homes and mobile computing devices. These VAAs would provide theemployees with services in their personal lives.

A DVAMS could define a first VDAE to include all the employees VAAs thatare associated with their corporate employment, and that are associatedwith Provider X. This would allow the DVAMS to make global changes tothe DVAMS services that the corporation provides to its employees. Acertain change could be made for all the employees by applying thechange to all of the elements defined in the first VDAE.

Note an employee would likely have both a first VAA from Provider X (forbusiness services) and a second VAA from provider Y (for personalservices) stored on the same device in his home. The employee could alsohave another VAA from Provider X (for business services) and another VAAfrom Provider Y (for personal services) stored on a device in hisoffice. When the DVAMS is instructed to make changes to the employee'swork related services, by altering only the VAAs that are in first VDAE,no changes will be made to the employees personal VAAs, even though theyare resident on the same devices.

Now, take the same basic fact pattern outlined above, and let's define asecond VDAE as encompassing all VAAs that are associated with a singleemployee. This would mean that all the work related VAAs from Provider Xand all the personal VAAs from Provider Y would be grouped in the secondVDAE. Now, if some aspect of personal information regarding the employeechanges, that change can be applied to all of the employees voiceservices by making the change for all elements in the second VDAE. Thechange could be applied regardless of who provides the services, andregardless of where the VAAs are located.

For instance, assume that the employee moves from New York City to LosAngeles. If this change in personal information is applied to all theelements in the second VDAE, this change would be reflected in all thework related VAAs from Provider X, and all personal VAAs from ProviderY. Thus, if the employees access the services to get the local weather,all of the VAAs, both work related and personal, would know to provide aweather forecast Los Angeles.

Likewise, a logical VDAE could be defined to include all VAAs, VAs andservices that are provided by a particular service provider. This wouldallow a DVAMS to make global changes to all elements of its system byapplying the change to all things grouped within that VDAE.

Or, a VDAE could include all users, devices and elements within acertain state. This would allow a DVAMS to apply the same change to allelements within the state. For instance, a change in the time caused bydaylight savings time.

A VAA can be a part of multiple different VDAEs, and may perform voiceapplications associated with a plurality of VDAEs. For instance, in theexamples given above, a particular employee's personal services VAA,which is provided by service provider Y, could be a part of a first VDAEthat associates all of that user's VAAs, and a part of a second VDAEthat associates all of the VAA's located within one state, and a part ofa third VDAE that associates all VAA's provided by service provider Y.

Any grouping of users, equipment, VAs, VAAs, VASSs, CDSs or other systemcomponents could be grouped into a VDAE by some common logicalconnection. And that VDAE grouping could then be used to help manage thesystem, and delivery of services to the users.

Each VA is allocated to at least one VDAE, which maps its deploymentacross one or more DVAES-enabled devices and VASS platforms that aresimilarly associated with the VDAE. The allocation is performed by aDVAMS as described below. Each VA to be deployed is allocated in thismanner. Once deployed, a VASS allocated to the VDAE “renders” the VA,producing a version of the VA components customized to operate withinthe constraints of each allocated VAA (and thus the devices that theVAAs are associated with). User associations within each VDAE mayprovide further information that is used to customize each VA withpersonalization information associated with a specific user or group ofusers.

A VDAE comprises associations between at least one DVAES-enabled device,at least one VASS, and at least one user. FIG. 17 a shows one exemplaryconfiguration, where a first VDAE comprises Device 15110 and VASS 15130(user and VAs not shown). FIG. 17 b shows an alternative exemplaryconfiguration, where second VDAE comprises Device 15110, Device 15120,and VASS 15130. As noted above, a VDAE can encompass any number ofdifferent system elements that are all tied together by some commonlogical connection.

A VDAE has users allocated to it after the user is registered with aDVAMS. In some instances, the allocation of users to a VDAE isautomatically performed on the basis of a default VDAE named in anaccount or device profile associated with the user's account or devicerespectively. Furthermore, users may be allocated to a VDAE from theoperator interface.

A VDAE has devices allocated to it because of relationships establishedbetween the devices and VAAs. Devices are rarely directly mapped to aspecific VDAE. Direct mapping of devices to one or more VDAEs isperformed in the operator interface.

A VDAE has VAAs allocated to it, based upon relationships between usersand devices. If a user and device are both associated with a VDAE, a VAAis associated with the user+device. If no VAA is presently associated, anew VDAE is created and is then associated with the user and device.

A VDAE has one or more VASSs associated with it. VASS's are associatedwith a VDAE based upon requests received from the operator interface.

A VDAE has one or more DVAMS's associated with it. DVAMS associationsare made based upon the operator interface.

A VDAE may have one or more CDS's associated with it. CDS associationsare made based upon the operator interface.

A VDAE has one or more VAs associated with it. Association between VAsand a VDAE is made an operator or user interface. VAs may be associatedwith a VDAE automatically if an allocated user has specified certain VAsas part of their preferences.

A VDAE may be used in several ways.

In one usage, a VDAE may be used to represent a group of users of aspecific device (e.g. a premise device). The VDAE represents the set ofusers, VAAs and VAs that are assigned to a particular premise device.

In another usage, a VDAE may represent the set of users, devices, andvoice applications that are managed by an operator.

In another usage, a VDAE could represent a social group, a workgroup atan office, members of an affinity group, members of a loyalty program(like a frequent flyer program), or members of a group that have signedup for a specific voice service.

VDAEs may be nested. Thus, a first VDAE may encompass or include aplurality of subordinate VDAEs. For example, a first VDAE may representa user's home. Multiple subordinate VDAEs might represent each familymember in the home. The VDAEs for each family member would beencompassed by or included within the first VDAE for the entire home.

As noted above, a single user may be associated with a plurality ofVDAEs, each of which represents a real world grouping of users. Forinstance, a first VDAE may be associated with said users' home device,and with voice applications appropriate for home use. A second VDAE maybe associated with the user's office premise device, and has voiceapplications appropriate for an office setting associated with it. Inthis situation, it may be possible to logically join the user to bothVDAE's, using the DVAMS, and having done this, the user becomes able toperform his business voice applications on his home device.

One or more VDAEs may be deployed to one or more DVAESs. A VDAE isdeployed by translating a VDAE specification into a deploymentspecification. The resulting deployment specification names the VAs tobe deployed to specific devices for use of specific users. For example,if a VDAE associates a first user and a second user with a first device,and further associates a first VA with said first user, and second andthird VAs with the second user, and determines a first VAA is present onthe first device, and a second VAA is present on a second device, adeployment specification that requires: 1. The first VA should berendered for the first user considering the environment of the first VAAand the first device; 2. The second and third VAs to be rendered for thesecond user should consider the environment of the second VAA and thesecond device; 3. The first VA components are distributed to the firstVAA; and 4. The second and third VA components are distributed to thesecond VAA.

A DVAMS can use the VDAEs to help update or upgrade system components.For instance, let's assume that a VDAE logically associates all users ofa particular VA. And let's assume that a voice dialing grammar in thatVA must be changed. In order to make this change, the VA must bere-rendered to all of the users who make use of the VA. Thesere-rendered VA components must then be propagated to all affectedDVAES-enabled devices.

The DVAMS can use the VDAE for mapping all of the users of the VA togenerate a deployment specification that lists all affected users and/orVAAs, and the DVAES-enabled equipment that uses the VA. The DVAMS wouldthen provide this deployment specification to the VASS and instruct theVASS to re-render the necessary VA components for all the users/VAAs inthe deployment specification. The VASS would then re-render thenecessary VA components for each of those users/VAAs listed in thedeployment specification. The VASS would also distribute the re-renderedVA components to the appropriate DVAES-enabled devices. This could bedone by notifying each of the affected DVAES-enabled devices to updatetheir caches with the newly tendered VA components.

The foregoing explained how VA components might be updated and sent toVAAs/devices. This process required the participation of the VASS tore-render the VA components. In other situations, the DVAMS mightdetermine that it is necessary to update the configuration materials ofa device. One embodiment of this sort of process, which is depicted inFIG. 18 , does not require the involvement of a VASS. In this process,the DVAMS creates the updated configuration materials and ensures thatthey are sent to the appropriate device.

In step 14110, a DVAMS determines that a particular device requires anupdate of one or more of its configuration materials. This determinationcould be based upon a change in the allocation of a device to a VDAE(and thus a DVAES). This determination could also be based uponreceiving notification of a change in required components on a device,or when a device is determined to require adjustments in itsconfiguration based upon performance, network topology changes, etc. Inother instances, the DVAMS may have completed an analysis of allocationmodels, configuration specifications, device performance reports, devicecapability information, or other materials and concluded that a changein a device's configuration materials is necessary.

In step 14120, the DVAMS generates the updated configuration materials.Next, in a first embodiment, in step 14130, the DVAMS pushes the updatedconfiguration materials to the CDS. In this first embodiment, the methodwould then proceed directly to step 14170, where the DVAMS would informthe cache manager on the VAA or device, via a communication protocol, torefresh the cache holding the configuration materials. The DVAMS mayprovide the destination for the refresh i.e. the CDS. Alternatively, thecache manager may know to go to the CDS based on update and refreshrules. Thus, the communication protocol between the DVAMS and theVAA/device may be specific, or it may be a general request to a cachemanager to obtain a non-cached version of the content.

In an alternate embodiment of the method, once the DVAMS has generatedthe new configuration materials, in step 14140, the updated materialsare cached somewhere on the network. Then, in step 14150 the DVAMSinforms the CDS via a communication protocol that there is new contentthat needs to be refreshed. Note this request can be a content requestmade to the CDS specifying delivery of a non-cached copy of the content,or it could be made via a CDS specific protocol/request. In step 14160the CDS fetches the content that needs to be refreshed. The content cancome either from a cache on the DVAMS, from a VASS, or from other DVAESlocations. The content that is typically stored on the CDS is common togroups of VAAs. This embodiment would then proceed to step 14170,discussed above, where the cache manager of a VAA or device isinstructed to update the cached configuration materials.

A DVAMS can also utilize clustering techniques to cluster VAAs, tothereby provide redundancy, distribute processing loads, and optimizethe use of specific system resources. In a first embodiment, VAAservices may be provided by a plurality of DVAES-enabled devices, withthe dispatch of specific voice sessions to any of a plurality of VAAinstances operating on disparate DVAES-enabled devices. By utilizing thevoice session transport mechanisms in this manner, VAA services may beprovided by whichever DVAES-enabled device is able to best provide therequested services at a specific point in time.

DVAES-enabled device's dynamic configuration mechanism supports theprovisioning of DVAES services using whichever device a user iscurrently accessing. In one aspect of the DVAES, a user may beprovisioned fully on a first DVAES-enabled device, and may be providedservices of said first device using any of a plurality of DVAES-enableddevices that are appropriately associated using one or more clusteringor cooperative processing techniques. Thus, a user may have access to atelephone device connected to a FSX port on a first DVAES-enableddevice, and be seamlessly connected over a network to their personalizedvoice applications deployed on a second DVAES-enabled device when theypick up the handset of the telephone. Similarly, a cluster ofDVAES-enabled devices may have user identification/authenticationmaterials deployed on each device within the cluster, and may route theuser's requests to one or more DVAES-enabled devices in the cluster forfulfillment. Said selection of services, and routing of requests, may beperformed upon the basis of aspects of the DVAES architecture and deviceloads, including, for example, specific device capabilities,provisioning decisions, current load, network latency, and devicelocation.

DVAES components may be aggregated in any desired manner and willinteroperate freely if appropriate credentials are provided. Sometimes,the aggregation takes the form of clustering. Clustering providesredundancy at the platform level and provides redundancy, and in someinstances, load balancing. In other aggregations, aggregation takes theform of cooperative processing where multiple hardware instances aremembers of a DVAES and each hardware instance may independently provideservices as required to perform a distributed, personalized voiceapplication for a user. The user receives services at whichever hardwaredevice they are using, without regard to the location that they areaccessing the DVAES from or the intervening network topology.

In addition, services may be provided on specific hardware platforms inwhich specialized hardware assistance is provided. For example, if aDVAES is implemented with a high-end server that provides complex speechrecognition, a VAA may be configured to attempt local speech recognitionon the local hardware, and to send complex or unrecognizable speech tothe high-end server for further analysis. Upon receipt of the resultsfrom the high-end speech recognition server, the VAA may continueprocessing the VA components locally.

The DVAES architecture supports a plurality of caching schemes. TheDVAES architecture optimizes the overall performance of the system byusing a combination of caching schemes, including the use of predictive,push and pull based caching, combined with the content distributionservice (CDS) technologies, and “wakeup-pull” caching schemes. Thecaching schemes may, in part, be based upon allocations, and thoseallocations may themselves be based upon VDAE groupings. Typically thecaching schemes would be rule based. These rules may be distributedwithin a DVAES as needed, and may be dynamically changed to account forvariations in network latency, processing capabilities, and usagepatterns. Traditional web-based content distribution networks (such asAkamai) are an additional caching mechanism that is advantageous to theDVAES architecture. Each of these caching techniques permits contentcreated by a VASS or DVAMS to be transparently propagated to aDVAES-enabled device.

Different cached items may have differing life spans. Some cached itemsmay be persistently stored in cache at a device, others may be stored ina CDS server and re-loaded by the device on a when needed/as neededbasis. Still other materials may be stored in their server of record,and be obtained as needed. Accordingly, DVAES caching is typically rulebased and is managed using caching rules stored as configurationmaterials applicable to a specific cache. Caching rules may be generatedby a DVAMS on the basis of network topology and each deployment'sperformance requirements.

Any reference in this specification to “one embodiment,” “anembodiment,” “example embodiment,” etc., means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of the invention. Theappearances of such phrases in various places in the specification arenot necessarily all referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with any embodiment, it is submitted that it is within thepurview of one skilled in the art to effect such feature, structure, orcharacteristic in connection with other ones of the embodiments.

Although the invention has been described with reference to a number ofillustrative embodiments thereof, it should be understood that numerousother modifications and embodiments can be devised by those skilled inthe art that will fall within the spirit and scope of the principles ofthis disclosure. More particularly, reasonable variations andmodifications are possible in the component parts and/or arrangements ofthe subject combination within the scope of the foregoing disclosure,the drawings and the appended claims without departing from the spiritof the invention. In addition to variations and modifications in thecomponent parts and/or arrangements, alternative uses will also beapparent to those skilled in the art.

What is claimed is:
 1. A distributed voice application execution system(DVAES)-enabled local device for providing voice services, comprising: amemory; at least one processor; an operating system; a networkinterface; an audio and signaling layer with at least one audio I/Odevice interface; and a voice applications agent (VAA) software clientconfigured to run on the at least one processor, the VAA software clientbeing further configured to register with a voice services provider(VSP) to have at least one credential that enables the VAA softwareclient to communicate with one or more servers of the VSP, wherein theVAA software client also is configured to start and manage a pluralityof runtime environments (RTEs), wherein each RTE can perform voiceapplication (VA) components and/or respond to requests and/orinstructions derived from VA components, and wherein the VAA softwareclient is configured to manage connectivity between the at least oneaudio I/O device interface and one or more RTEs; wherein theDVAES-enabled local device includes one or more configuration materialsand/or settings configured to be controlled by the VSP; wherein the VAAsoftware client is configured to: send, to one or more servers of theVSP, one or more messages that include information regarding an event,setting, state and/or variable associated with at least one component ofthe DVAES-enabled local device; receive, from one or more servers of theVSP, VA components for one or more voice applications, where the VAcomponents are based on events, settings, states and/or variablesassociated with components of the local device; and manage one or moreRTEs as the RTEs perform VA components and/or respond to requests and/orinstructions derived from VA components.
 2. The device of claim 1,wherein the one or more messages include information regarding an event,setting, state and/or variable associated with the VAA software client.3. The device of claim 1, wherein the one or more messages includeinformation regarding an event, setting, state and/or variableassociated with a RTE managed by the VAA software client.
 4. The deviceof claim 1, wherein the one or more messages include informationregarding an event, setting, state and/or variable associated with a VAcomponent.
 5. The device of claim 1, wherein to manage one or more RTEscomprises the VAA software client causing at least one of the one ormore RTEs to perform a VA component and/or respond to requests and/orinstructions derived from a VA component according to established logic.6. The device of claim 5, wherein the established logic comprises logicincluded in one or more VA components.
 7. The device of claim 1, whereinthe at least one audio I/O device interface comprises a first I/O deviceinterface, and wherein to manage one or more RTEs comprises the VAAsoftware client controlling which of the one or more RTEs is connectedto the first audio I/O device interface.
 8. The device of claim 7,wherein the at least one audio I/O device interface further comprises asecond audio I/O device interface that is functionally coupled to anaudio device that is separate from the DVAES-enabled local device, andwherein to manage one or more RTEs comprises the VAA software clientcontrolling which of the one or more RTEs is connected to the secondaudio I/O device interface.
 9. The device of claim 1, wherein at leastsome of the plurality of RTEs are configured by the VAA software clientto perform a predetermined function or set of functions.
 10. The deviceof claim 1, wherein the VAA software client is also configured to starta plurality of RTEs when the VAA software client starts.
 11. The deviceof claim 1, wherein the VAA software client also is configured to sendupdate messages to one or more servers of the VSP as the VAA softwareclient manages at least one of the one or more RTEs as the at least oneRTE performs a VA component or responds to requests and/or instructionsderived from a VA component, wherein each update message includesinformation regarding an event, setting, state and/or variableassociated with a component of the DVAES-enabled local device.
 12. Thedevice of claim 1, wherein the VAA software client is further configuredto receive at least one VA component that has been rendered based on acapability or characteristic of the DVAES-enabled local device.
 13. Thedevice of claim 1, wherein the VAA software client is further configuredto receive at least one VA component that has been rendered based on acapability or characteristic of a data network connection of theDVAES-enabled local device.
 14. The device of claim 1, wherein the VAAsoftware client is further configured to receive at least one VAcomponent that has been rendered based on a characteristic of a user ofthe DVAES-enabled local device.
 15. The device of claim 1, wherein theVAA software client is further configured to receive VA components forfirst and second voice applications.
 16. The device of claim 15, whereinthe VAA software client is configured to manage the one or more RTEs asthe one or more RTEs perform VA components for the first and secondvoice applications and/or respond to requests and/or instructionsderived from VA components for the first and second voice applicationssuch that the first and second voice applications are performedconcurrently.
 17. The device of claim 1, wherein the VAA software clientis further configured to: determine that a new RTE is needed to performa particular received VA component or to respond to a request and/orinstruction derived from the particular VA component; and start a newRTE that is to be used to perform the particular received VA componentand/or to respond to a request and/or instruction derived from theparticular VA component, and wherein to manage further comprises the VAAsoftware client managing the new RTE as the new RTE performs theparticular VA component and/or responds to requests and/or instructionsderived from the particular VA component.
 18. The device of claim 17,wherein determining that a new RTE is needed to perform the particularreceived VA component or to respond to a request and/or instructionderived from the particular VA component comprises determining that noneof the RTEs currently being managed by the VAA software client areavailable for and/or are capable of performing the particular receivedVA component and/or of responding to a request and/or instructionderived from the particular VA component.
 19. The device of claim 1,wherein to manage one or more RTEs comprises the VAA software client:pausing performance of VA components of a first voice application uponoccurrence of a trigger event; causing VA components of a second voiceapplication to be performed once performance of VA components of thefirst voice application has been paused; and resuming performance of theVA components of the first voice application when performance of the VAcomponents of the second voice application is completed.
 20. The deviceof claim 1, wherein the at least one audio I/O device interfacecomprises first and second audio I/O device interfaces, and wherein tomanage one or more RTEs comprises the VAA software client causing afirst RTE that is connected to the first audio I/O device interface toperform a first VA component and/or respond to a request and/orinstruction derived from the first VA component and also causing asecond RTE that is connected to the second audio I/O device interface toperform a second VA component and/or respond to a request and/orinstruction derived from the second VA component.
 21. The device ofclaim 1, wherein the VAA software client is configured to detect when auser speaks a keyword or phrase by: continuously monitoring soundsreceived via a microphone coupled to the at least one audio I/O deviceinterface; detecting when the received sounds include the keyword orphrase; and taking a predetermined action when the user has spoken thekeyword or phrase.
 22. The device of claim 21, wherein taking apredetermined action comprises: receiving audio input after detection ofthe keyword or phrase; and sending the received audio to a speechrecognition interface resident on the DVAES-enabled local device or to aserver of the VSP.
 23. The device of claim 1, wherein the VAA softwareclient is further configured to send to a server of the VSP a setting,state or variable of one or more configuration materials or settings ofa component of the DVAES-enabled local device.
 24. The device of claim1, wherein the VAA software client is further configured to send to aserver of the VSP a setting, state or variable of one or moreconfiguration materials or settings of the VAA software client.
 25. Thedevice of claim 1, wherein the VAA software client is configured to sendto a server of the VSP device log messages relating to operations of theDVAES-enabled local device.
 26. The device of claim 1, wherein the VAAsoftware client is configured to send to a server of the VSP logmessages and/or error messages relating to operations of the VAAsoftware client.
 27. The device of claim 1, wherein the DVAES-enabledlocal device includes configuration materials that control how theDVAES-enabled local device performs device startup procedures.
 28. Thedevice of claim 1, wherein the one or more messages include values ofmultiple variables as of a specific time, and wherein the values ofmultiple variables relate to at least one of a component of theDVAES-enabled local device and performance of a VA component.
 29. Thedevice of claim 1, wherein the VAA software client is further configuredto send to a server of the VSP the at least one credential of the VAAsoftware client, and wherein the server of the VSP is configured to usethe at least one credential to uniquely identify the VAA softwareclient, to uniquely identify the DVAES-enabled local device, to obtaininformation associated with the VAA software client and/or to obtaininformation associated with the DVAES-enabled local device.
 30. Thedevice of claim 1, wherein the VAA software client is further configuredto send to a server of the VSP the at least one credential of the VAAsoftware client, and wherein the server of the VSP is configured to usethe at least one credential to identify a user account associated withthe DVAES-enabled local device and to obtain information associated withthat user account.